Content adaptive prediction and entropy coding of motion vectors for next generation video

ABSTRACT

Techniques related to content adaptive prediction and entropy coding of motion vectors are described.

RELATED APPLICATIONS

The present application is a divisional of U.S. patent application Ser.No. 14/334,257, filed Jul. 17, 2014, which is a continuation-in-part ofInternational Application No. PCT/US2013/077698, filed Dec. 24, 2013,which claims the benefit of U.S. Provisional Application No. 61/758,314filed Jan. 30, 2013, the contents of which are expressly incorporatedherein in their entireties.

BACKGROUND

A video encoder compresses video information so that more informationcan be sent over a given bandwidth. The compressed signal may then betransmitted to a receiver having a decoder that decodes or decompressesthe signal prior to display.

High Efficient Video Coding (HEVC) is the latest video compressionstandard, which is being developed by the Joint Collaborative Team onVideo Coding (JCT-VC) formed by ISO/IEC Moving Picture Experts Group(MPEG) and ITU-T Video Coding Experts Group (VCEG). HEVC is beingdeveloped in response to the previous H.264/AVC (Advanced Video Coding)standard not providing enough compression for evolving higher resolutionvideo applications. Similar to previous video coding standards, HEVCincludes basic functional modules such as intra/inter prediction,transform, quantization, in-loop filtering, and entropy coding.

The ongoing HEVC standard may attempt to improve on limitations of theH.264/AVC standard such as limited choices for allowed predictionpartitions and coding partitions, limited allowed multiple referencesand prediction generation, limited transform block sizes and actualtransforms, limited mechanisms for reducing coding artifacts, andinefficient entropy encoding techniques. However, the ongoing HEVCstandard may use iterative approaches to solving such problems.

For instance, with ever increasing resolution of video to be compressedand expectation of high video quality, the correspondingbitrate/bandwidth required for coding using existing video codingstandards such as H.264 or even evolving standards such as H.265/HEVC,is relatively high. The aforementioned standards use expanded forms oftraditional approaches to implicitly address the insufficientcompression/quality problem, but often the results are limited.

The present description, developed within the context of a NextGeneration Video (NGV) codec project, addresses the general problem ofdesigning an advanced video codec that maximizes the achievablecompression efficiency while remaining sufficiently practical forimplementation on devices. For instance, with ever increasing resolutionof video and expectation of high video quality due to availability ofgood displays, the corresponding bitrate/bandwidth required usingexisting video coding standards such as earlier MPEG standards and eventhe more recent H.264/AVC standard, is relatively high. H.264/AVC wasnot perceived to be providing high enough compression for evolvinghigher resolution video applications.

BRIEF DESCRIPTION OF THE DRAWINGS

The material described herein is illustrated by way of example and notby way of limitation in the accompanying figures. For simplicity andclarity of illustration, elements illustrated in the figures are notnecessarily drawn to scale. For example, the dimensions of some elementsmay be exaggerated relative to other elements for clarity. Further,where considered appropriate, reference labels have been repeated amongthe figures to indicate corresponding or analogous elements. In thefigures:

FIG. 1 is an illustrative diagram of an example next generation videoencoder;

FIG. 2 is an illustrative diagram of an example next generation videodecoder;

FIG. 3(a) is an illustrative diagram of an example next generation videoencoder and subsystems;

FIG. 3(b) is an illustrative diagram of an example next generation videodecoder and subsystems;

FIG. 4 is an illustrative diagram of an example entropy encoder module;

FIG. 5 is an illustrative diagram of an example entropy decoder module;

FIG. 6 is an illustrative diagram of an example entropy encoder module;

FIG. 7 is an illustrative diagram of an example entropy decoder module;

FIG. 8 is a flow diagram illustrating an example process;

FIG. 9 illustrates an example bitstream;

FIG. 10 is a flow diagram illustrating an example process;

FIG. 11 is an illustrative diagram of an example video coding system andvideo coding process in operation;

FIG. 12 is a flow diagram illustrating an example encoding process;

FIG. 13 illustrates an example bitstream;

FIG. 14 is a flow diagram illustrating an example decoding process;

FIGS. 15(A), 15(B), and 15(C) provide an illustrative diagram of anexample video coding system and video coding process in operation;

FIG. 16 is an illustrative diagram of an example video coding system;

FIG. 17 is an illustrative diagram of an example system;

FIG. 18 illustrates an example device;

FIG. 19 is an illustrative diagram of an example motion vector anddifferential motion vector encoder module;

FIG. 20 is an illustrative diagram of an example video block andneighboring blocks;

FIG. 21 is an illustrative diagram of an example video block andneighboring blocks for use in example motion vector predictiontechniques;

FIG. 22 is an illustrative diagram of an example video block andneighboring blocks for use in example motion vector predictiontechniques;

FIG. 23 is an illustrative diagram of an example video block andneighboring blocks for use in example motion vector predictiontechniques;

FIG. 24 is a flow diagram illustrating an example process;

FIG. 25 is a flow diagram illustrating an example process;

FIG. 26 is an illustrative diagram of an example adaptive motionprediction analyzer module;

FIGS. 27(A)-(D) provide a flow diagram illustrating an example process;

FIG. 28 is an illustrative diagram of an example adaptive VLC encoderfor motion vector differences module;

FIG. 29 is an illustrative diagram of an example differential motionvector classes; and

FIGS. 30(A) and 30(B) are illustrative diagrams of example differentialmotion vector subclasses, all arranged in accordance with at least someimplementations of the present disclosure.

DETAILED DESCRIPTION

One or more embodiments or implementations are now described withreference to the enclosed figures. While specific configurations andarrangements are discussed, it should be understood that this is donefor illustrative purposes only. Persons skilled in the relevant art willrecognize that other configurations and arrangements may be employedwithout departing from the spirit and scope of the description. It willbe apparent to those skilled in the relevant art that techniques and/orarrangements described herein may also be employed in a variety of othersystems and applications other than what is described herein.

While the following description sets forth various implementations thatmay be manifested in architectures such as system-on-a-chip (SoC)architectures for example, implementation of the techniques and/orarrangements described herein are not restricted to particulararchitectures and/or computing systems and may be implemented by anyarchitecture and/or computing system for similar purposes. For instance,various architectures employing, for example, multiple integratedcircuit (IC) chips and/or packages, and/or various computing devicesand/or consumer electronic (CE) devices such as set top boxes, smartphones, etc., may implement the techniques and/or arrangements describedherein. Further, while the following description may set forth numerousspecific details such as logic implementations, types andinterrelationships of system components, logic partitioning/integrationchoices, etc., claimed subject matter may be practiced without suchspecific details. In other instances, some material such as, forexample, control structures and full software instruction sequences, maynot be shown in detail in order not to obscure the material disclosedherein.

The material disclosed herein may be implemented in hardware, firmware,software, or any combination thereof. The material disclosed herein mayalso be implemented as instructions stored on a machine-readable medium,which may be read and executed by one or more processors. Amachine-readable medium may include any medium and/or mechanism forstoring or transmitting information in a form readable by a machine(e.g., a computing device). For example, a machine-readable medium mayinclude read only memory (ROM); random access memory (RAM); magneticdisk storage media; optical storage media; flash memory devices;electrical, optical, acoustical or other forms of propagated signals(e.g., carrier waves, infrared signals, digital signals, etc.), andothers.

References in the specification to “one implementation”, “animplementation”, “an example implementation”, etc., indicate that theimplementation described may include a particular feature, structure, orcharacteristic, but every embodiment may not necessarily include theparticular feature, structure, or characteristic. Moreover, such phrasesare not necessarily referring to the same implementation. Further, whena particular feature, structure, or characteristic is described inconnection with an embodiment, it is submitted that it is within theknowledge of one skilled in the art to effect such feature, structure,or characteristic in connection with other implementations whether ornot explicitly described herein.

Systems, apparatus, articles, and methods are described below related tocontent adaptive entropy coding for video systems.

Next generation video (NGV) systems, apparatus, articles, and methodsare described below. NGV video coding may incorporate significantcontent based adaptivity in the video coding process to achieve highercompression. As discussed above, the H.264/AVC standard may have avariety of limitations and ongoing attempts to improve on the standard,such as, for example, the HEVC standard may use iterative approaches toaddress such limitations. Herein, an NGV system including an encoder anda decoder will be described.

For example, the H.264 standard may support two modes of entropy coding.In the first mode, Adaptive VLC (Variable Length Coding), transformcoefficients may be coded using Context Adaptive VLC (CAVLC) and allother syntax elements (e.g., overhead data, modes, motion vectors, andso on) may be coded using Exponential Golomb coding. The CAVLC codeddata and the Exponential Golomb coded data may be multiplexed togenerate an encoded bitstream. In the second mode, all data may beencoded using Context Adaptive Binary Arithmetic Coding (CABAC). Thecorresponding decoder may also operate in two modes, disassembling anddecoding the multiplexed bit stream in the first mode and decoding theCABAC encoded bitstream in the second mode. Such techniques may havelimitations. For example, CABAC coding may be efficient but may becomplex such that throughput in higher resolution contexts may belimited. Further, by grouping data types together for coding, efficiencyand complexity of the entropy coding may not be optimized.

In some video codec implementations, entropy coding and decoding ofvarious data such as overhead data, modes, motion vectors, and/ortransform coefficients may be a significant contributor to the codingefficiency and complexity of the system. In some examples, thetechniques discussed herein may balance coding efficiency and systemcomplexity.

In some examples, first video data and second video data may be receivedfor entropy encoding at an entropy encoder module. The first video dataand the second video data may be different data types (e.g., headerdata, morphing parameters, synthesizing parameters, or global maps dataor motion vectors or intra-prediction partition data or so on, as isdiscussed further herein). A first entropy encoding technique may bedetermined for the first video data based on a parameter associated withthe first video data such as, for example, a number of compressed bitsof the first video data, a predetermined indicator or flag associatedwith the first video data, a predetermined threshold, or a heuristicallydetermined threshold or the like. In some examples, the first entropyencoding technique may be chosen from one of an adaptive symbol-runvariable length coding technique or an adaptive proxy variable lengthcoding technique. The first video data may be entropy encoded using thefirst entropy encoding technique and the second video data may beentropy encoded using the first entropy encoding technique. In someexamples, the second entropy encoding technique may be determined forthe second video data based on a parameter as discussed with respect tothe first video data. In some examples, the second entropy encodingtechnique may be determined from options including an adaptivesymbol-run variable length coding technique, an adaptive proxy variablelength coding technique, an adaptive vector variable length coding, anadaptive 1-dimensional variable length coding technique, an adaptive2-dimensional variable length coding technique, or other techniques asdiscussed herein. In some examples, the second entropy encodingtechnique may be predetermined for the data type of the second videodata. Entropy encoding the first video data and the second video datamay generate first compressed video data and second compressed videodata. The first and second compressed video data may be assembled togenerate an output bitstream.

The output bitstream may be transmitted from the encoder to a decoder,which may disassemble the bitstream to determine the first and secondcompressed video data. The compressed video data may be entropy decodedto generate entropy decoded video data, which may be further decoded togenerate a video frame. The video frame may be transmitted to a displaydevice for presentment to a user.

In some examples, additional video data types may be received forentropy encoding at the entropy encoder module. For example, third,fourth, fifth, sixth, seventh, or more video data may be entropy encodedto generate associated compressed video data, which may be assembled inthe output bitstream, transmitted, and subsequently entropy decoded viaa decoder. The various data types and encoding/decoding techniqueoptions and implementations are discussed further herein.

As used herein, the term “coder” may refer to an encoder and/or adecoder. Similarly, as used herein, the term “coding” may refer toperforming video encoding via an encoder and/or performing videodecoding via a decoder. For example a video encoder and video decodermay both be examples of coders capable of coding video data. Inaddition, as used herein, the term “codec” may refer to any process,program or set of operations, such as, for example, any combination ofsoftware, firmware, and/or hardware, that may implement an encoderand/or a decoder. Further, as used herein, the phrase “video data” mayrefer to any type of data associated with video coding such as, forexample, video frames, image data, encoded bitstream data, or the like.

FIG. 1 is an illustrative diagram of an example next generation videoencoder 100, arranged in accordance with at least some implementationsof the present disclosure. As shown, encoder 100 may receive input video101. Input video 101 may include any suitable input video for encodingsuch as, for example, input frames of a video sequence. As shown, inputvideo 101 may be received via a content pre-analyzer module 102. Contentpre-analyzer module 102 may be configured to perform analysis of thecontent of video frames of input video 101 to determine various types ofparameters for improving video coding efficiency and speed performance.For example, content pre-analyzer module 102 may determine horizontaland vertical gradient information (e.g., Rs, Cs), variance, spatialcomplexity per picture, temporal complexity per picture, scene changedetection, motion range estimation, gain detection, prediction distanceestimation, number of objects estimation, region boundary detection,spatial complexity map computation, focus estimation, film grainestimation, or the like. The parameters generated by contentpre-analyzer module 102 may be used by encoder 100 (e.g., via encodecontroller 103) and/or quantized and communicated to a decoder. Asshown, video frames and/or other data may be transmitted from contentpre-analyzer module 102 to adaptive picture organizer module 104, whichmay determine the picture type (e.g., I-, P-, or F/B-picture) of eachvideo frame and reorder the video frames as needed. In some examples,adaptive picture organizer module 104 may include a frame portiongenerator configured to generate frame portions. In some examples,content pre-analyzer module 102 and adaptive picture organizer module104 may together be considered a pre-analyzer subsystem of encoder 100.

As shown, video frames and/or other data may be transmitted fromadaptive picture organizer module 104 to prediction partitions generatormodule 105. In some examples, prediction partitions generator module 105may divide a frame or picture into tiles or super-fragments or the like.In some examples, an additional module (e.g., between modules 104 and105) may be provided for dividing a frame or picture into tiles orsuper-fragments. Prediction partitions generator module 105 may divideeach tile or super-fragment into potential prediction partitionings orpartitions. In some examples, the potential prediction partitionings maybe determined using a partitioning technique such as, for example, a k-dtree partitioning technique, a bi-tree partitioning technique, or thelike, which may be determined based on the picture type (e.g., I-, P-,or F/B-picture) of individual video frames, a characteristic of theframe portion being partitioned, or the like. In some examples, thedetermined potential prediction partitionings may be partitions forprediction (e.g., inter- or intra-prediction) and may be described asprediction partitions or prediction blocks or the like.

In some examples, a selected prediction partitioning (e.g., predictionpartitions) may be determined from the potential predictionpartitionings. For example, the selected prediction partitioning may bebased on determining, for each potential prediction partitioning,predictions using characteristics and motion based multi-referencepredictions or intra-predictions, and determining prediction parameters.For each potential prediction partitioning, a potential prediction errormay be determined by differencing original pixels with prediction pixelsand the selected prediction partitioning may be the potential predictionpartitioning with the minimum prediction error. In other examples, theselected prediction partitioning may be determined based on a ratedistortion optimization including a weighted scoring based on number ofbits for coding the partitioning and a prediction error associated withthe prediction partitioning.

As shown, the original pixels of the selected prediction partitioning(e.g., prediction partitions of a current frame) may be differenced withpredicted partitions (e.g., a prediction of the prediction partition ofthe current frame based on a reference frame or frames and otherpredictive data such as inter- or intra-prediction data) at differencer106. The determination of the predicted partitions will be describedfurther below and may include a decode loop as shown in FIG. 1. Anyresiduals or residual data (e.g., partition prediction error data) fromthe differencing may be transmitted to coding partitions generatormodule 107. In some examples, such as for intra-prediction of predictionpartitions in any picture type (I-, F/B- or P-pictures), codingpartitions generator module 107 may be bypassed via switches 107 a and107 b. In such examples, only a single level of partitioning may beperformed. Such partitioning may be described as prediction partitioning(as discussed) or coding partitioning or both. In various examples, suchpartitioning may be performed via prediction partitions generator module105 (as discussed) or, as is discussed further herein, such partitioningmay be performed via a k-d tree intra-prediction/coding partitionermodule or a bi-tree intra-prediction/coding partitioner moduleimplemented via coding partitions generator module 107.

In some examples, the partition prediction error data, if any, may notbe significant enough to warrant encoding. In other examples, where itmay be desirable to encode the partition prediction error data and thepartition prediction error data is associated with inter-prediction orthe like, coding partitions generator module 107 may determine codingpartitions of the prediction partitions. In some examples, codingpartitions generator module 107 may not be needed as the partition maybe encoded without coding partitioning (e.g., as shown via the bypasspath available via switches 107 a and 107 b). With or without codingpartitioning, the partition prediction error data (which maysubsequently be described as coding partitions in either event) may betransmitted to adaptive transform module 108 in the event the residualsor residual data require encoding. In some examples, predictionpartitions generator module 105 and coding partitions generator module107 may together be considered a partitioner subsystem of encoder 100.In various examples, coding partitions generator module 107 may operateon partition prediction error data, original pixel data, residual data,or wavelet data.

Coding partitions generator module 107 may generate potential codingpartitionings (e.g., coding partitions) of, for example, partitionprediction error data using bi-tree and/or k-d tree partitioningtechniques or the like. In some examples, the potential codingpartitions may be transformed using adaptive or fixed transforms withvarious block sizes via adaptive transform module 108 and a selectedcoding partitioning and selected transforms (e.g., adaptive or fixed)may be determined based on a rate distortion optimization or otherbasis. In some examples, the selected coding partitioning and/or theselected transform(s) may be determined based on a predeterminedselection method based on coding partitions size or the like.

For example, adaptive transform module 108 may include a first portionor component for performing a parametric transform to allow locallyoptimal transform coding of small to medium size blocks and a secondportion or component for performing globally stable, low overheadtransform coding using a fixed transform, such as a discrete cosinetransform (DCT) or a picture based transform from a variety oftransforms, including parametric transforms, or any other configurationas is discussed further herein. In some examples, for locally optimaltransform coding a Parametric Haar Transform (PHT) may be performed, asis discussed further herein. In some examples, transforms may beperformed on 2D blocks of rectangular sizes between about 4×4 pixels and64×64 pixels, with actual sizes depending on a number of factors such aswhether the transformed data is luma or chroma, or inter or intra, or ifthe determined transform used is PHT or DCT or the like.

As shown, the resultant transform coefficients may be transmitted toadaptive quantize module 109. Adaptive quantize module 109 may quantizethe resultant transform coefficients. Further, any data associated witha parametric transform, as needed, may be transmitted to either adaptivequantize module 109 (if quantization is desired) or adaptive entropyencoder module 110. Also as shown in FIG. 1, the quantized coefficientsmay be scanned and transmitted to adaptive entropy encoder module 110.Adaptive entropy encoder module 110 may entropy encode the quantizedcoefficients and include them in output bitstream 111. In some examples,adaptive transform module 108 and adaptive quantize module 109 maytogether be considered a transform encoder subsystem of encoder 100.

As also shown in FIG. 1, encoder 100 includes a local decode loop. Thelocal decode loop may begin at adaptive inverse quantize module 112.Adaptive inverse quantize module 112 may be configured to perform theopposite operation(s) of adaptive quantize module 109 such that aninverse scan may be performed and quantized coefficients may bede-scaled to determine transform coefficients. Such an adaptive quantizeoperation may be lossy, for example. As shown, the transformcoefficients may be transmitted to an adaptive inverse transform module113. Adaptive inverse transform module 113 may perform the inversetransform as that performed by adaptive transform module 108, forexample, to generate residuals or residual values or partitionprediction error data (or original data or wavelet data, as discussed)associated with coding partitions. In some examples, adaptive inversequantize module 112 and adaptive inverse transform module 113 maytogether be considered a transform decoder subsystem of encoder 100.

As shown, the partition prediction error data (or the like) may betransmitted to optional coding partitions assembler 114. Codingpartitions assembler 114 may assemble coding partitions into decodedprediction partitions as needed (as shown, in some examples, codingpartitions assembler 114 may be skipped via switches 114 a and 114 bsuch that decoded prediction partitions may have been generated atadaptive inverse transform module 113) to generate prediction partitionsof prediction error data or decoded residual prediction partitions orthe like.

As shown, the decoded residual prediction partitions may be added topredicted partitions (e.g., prediction pixel data) at adder 115 togenerate reconstructed prediction partitions. The reconstructedprediction partitions may be transmitted to prediction partitionsassembler 116. Prediction partitions assembler 116 may assemble thereconstructed prediction partitions to generate reconstructed tiles orsuper-fragments. In some examples, coding partitions assembler module114 and prediction partitions assembler module 116 may together beconsidered an un-partitioner subsystem of encoder 100.

The reconstructed tiles or super-fragments may be transmitted toblockiness analyzer and deblock filtering module 117. Blockinessanalyzer and deblock filtering module 117 may deblock and dither thereconstructed tiles or super-fragments (or prediction partitions oftiles or super-fragments). The generated deblock and dither filterparameters may be used for the current filter operation and/or coded inoutput bitstream 111 for use by a decoder, for example. The output ofblockiness analyzer and deblock filtering module 117 may be transmittedto a quality analyzer and quality restoration filtering module 118.Quality analyzer and quality restoration filtering module 118 maydetermine QR filtering parameters (e.g., for a QR decomposition) and usethe determined parameters for filtering. The QR filtering parameters mayalso be coded in output bitstream 111 for use by a decoder. As shown,the output of quality analyzer and quality restoration filtering module118 may be transmitted to decoded picture buffer 119. In some examples,the output of quality analyzer and quality restoration filtering module118 may be a final reconstructed frame that may be used for predictionfor coding other frames (e.g., the final reconstructed frame may be areference frame or the like). In some examples, blockiness analyzer anddeblock filtering module 117 and quality analyzer and qualityrestoration filtering module 118 may together be considered a filteringsubsystem of encoder 100.

In encoder 100, prediction operations may include inter- and/orintra-prediction. As shown in FIG. 1(a), inter-prediction may beperformed by one or more modules including morphing analyzer and morphedpicture generation module 120, synthesizing analyzer and generationmodule 121, and characteristics and motion filtering predictor module123. Morphing analyzer and morphed picture generation module 120 mayanalyze a current picture to determine parameters for changes in gain,changes in dominant motion, changes in registration, and changes in blurwith respect to a reference frame or frames with which it may be to becoded. The determined morphing parameters may be quantized/de-quantizedand used (e.g., by morphing analyzer and morphed picture generationmodule 120) to generate morphed reference frames that that may be usedby motion estimator module 122 for computing motion vectors forefficient motion (and characteristics) compensated prediction of acurrent frame. Synthesizing analyzer and generation module 121 maygenerate super resolution (SR) pictures and projected interpolation (PI)pictures or the like for motion for determining motion vectors forefficient motion compensated prediction in these frames.

Motion estimator module 122 may generate motion vector data based onmorphed reference frame(s) and/or super resolution (SR) pictures andprojected interpolation (PI) pictures along with the current frame. Insome examples, motion estimator module 122 may be considered aninter-prediction module. For example, the motion vector data may be usedfor inter-prediction. If inter-prediction is applied, characteristicsand motion compensated filtering predictor module 123 may apply motioncompensation as part of the local decode loop as discussed.

Intra-prediction may be performed by intra-directional predictionanalyzer and prediction generation module 124. Intra-directionalprediction analyzer and prediction generation module 124 may beconfigured to perform spatial directional prediction and may use decodedneighboring partitions. In some examples, both the determination ofdirection and generation of prediction may be performed byintra-directional prediction analyzer and prediction generation module124. In some examples, intra-directional prediction analyzer andprediction generation module 124 may be considered an intra-predictionmodule.

As shown in FIG. 1, prediction modes and reference types analyzer module125 may allow for selection of prediction modes from among, “skip”,“auto”, “inter”, “split”, “multi”, and “intra”, for each predictionpartition of a tile (or super-fragment), all of which may apply to P-and F/B-pictures. In addition to prediction modes, it also allows forselection of reference types that can be different depending on “inter”or “multi” mode, as well as for P- and F/B-pictures. The predictionsignal at the output of prediction modes and reference types analyzermodule 125 may be filtered by prediction analyzer and prediction fusionfiltering module 126. Prediction analyzer and prediction fusionfiltering module 126 may determine parameters (e.g., filteringcoefficients, frequency, overhead) to use for filtering and may performthe filtering. In some examples, filtering the prediction signal mayfuse different types of signals representing different modes (e.g.,intra, inter, multi, split, skip, and auto). In some examples,intra-prediction signals may be different than all other types ofinter-prediction signal(s) such that proper filtering may greatlyenhance coding efficiency. In some examples, the filtering parametersmay be encoded in output bitstream 111 for use by a decoder. Thefiltered prediction signal may provide the second input (e.g.,prediction partition(s)) to differencer 106, as discussed above, thatmay determine the prediction difference signal (e.g., partitionprediction error) for coding discussed earlier. Further, the samefiltered prediction signal may provide the second input to adder 115,also as discussed above. As discussed, output bitstream 111 may providean efficiently encoded bitstream for use by a decoder for thepresentment of video.

FIG. 1 illustrates example control signals associated with operation ofvideo encoder 100, where the following abbreviations may represent theassociated information:

-   -   scnchg Scene change information    -   spcpx Spatial complexity information    -   tpcpx Temporal complexity information    -   pdist Temporal prediction distance information    -   pap Pre Analysis parameters (placeholder for all other pre        analysis parameters except scnchg, spcpx, tpcpx, pdist)    -   ptyp Picture types information    -   pgst Picture group structure information    -   pptn cand. Prediction partitioning candidates    -   cptn cand. Coding Partitioning Candidates    -   prp Preprocessing    -   xmtyp Transform type information    -   xmdir Transform direction information    -   xmmod Transform mode    -   ethp One eighth (1/8th) pel motion prediction    -   pptn Prediction Partitioning    -   cptn Coding Partitioning    -   mot&cod cost Motion and Coding Cost    -   qs quantizer information set (includes Quantizer parameter (Qp),        Quantizer matrix (QM) choice)    -   my Motion vectors    -   mop Morphing Paramters    -   syp Synthesizing Parameters    -   ddi Deblock and dither information    -   qri Quality Restoration filtering index/information    -   api Adaptive Precision filtering index/information    -   flu Fusion Filtering index/information    -   mod Mode information    -   reftyp Reference type information    -   idir Intra Prediction Direction

The various signals and data items that may need to be sent to thedecoder, ie, pgst, ptyp, prp, pptn, cptn, modes, reftype, ethp, xmtyp,xmdir, , xmmod, idir, my, qs, mop, syp, ddi, qri, api, fh, quantcoefficients and others may then be entropy encoded by adaptive entropyencoder 110 that may include different entropy coders collectivelyreferred to as an entropy encoder subsystem. While these control signalsare illustrated as being associated with specific example functionalmodules of encoder 100 in FIG. 1, other implementations may include adifferent distribution of control signals among the functional modulesof encoder 300. The present disclosure is not limited in this regardand, in various examples, implementation of the control signals hereinmay include the undertaking of only a subset of the specific examplecontrol signals shown, additional control signals, and/or in a differentarrangement than illustrated.

FIG. 2 is an illustrative diagram of an example next generation videodecoder 200, arranged in accordance with at least some implementationsof the present disclosure. As shown, decoder 200 may receive an inputbitstream 201. In some examples, input bitstream 201 may be encoded viaencoder 100 and/or via the encoding techniques discussed herein. Asshown, input bitstream 201 may be received by an adaptive entropydecoder module 202. Adaptive entropy decoder module 202 may decode thevarious types of encoded data (e.g., overhead, motion vectors, transformcoefficients, etc.). In some examples, adaptive entropy decoder 202 mayuse a variable length decoding technique. In some examples, adaptiveentropy decoder 202 may perform the inverse operation(s) of adaptiveentropy encoder module 110 discussed above.

The decoded data may be transmitted to adaptive inverse quantize module203. Adaptive inverse quantize module 203 may be configured to inversescan and de-scale quantized coefficients to determine transformcoefficients. Such an adaptive quantize operation may be lossy, forexample. In some examples, adaptive inverse quantize module 203 may beconfigured to perform the opposite operation of adaptive quantize module109 (e.g., substantially the same operations as adaptive inversequantize module 112). As shown, the transform coefficients (and, in someexamples, transform data for use in a parametric transform) may betransmitted to an adaptive inverse transform module 204. Adaptiveinverse transform module 204 may perform an inverse transform on thetransform coefficients to generate residuals or residual values orpartition prediction error data (or original data or wavelet data)associated with coding partitions. In some examples, adaptive inversetransform module 204 may be configured to perform the opposite operationof adaptive transform module 108 (e.g., substantially the sameoperations as adaptive inverse transform module 113). In some examples,adaptive inverse transform module 204 may perform an inverse transformbased on other previously decoded data, such as, for example, decodedneighboring partitions. In some examples, adaptive inverse quantizemodule 203 and adaptive inverse transform module 204 may together beconsidered a transform decoder subsystem of decoder 200.

As shown, the residuals or residual values or partition prediction errordata may be transmitted to coding partitions assembler 205. Codingpartitions assembler 205 may assemble coding partitions into decodedprediction partitions as needed (as shown, in some examples, codingpartitions assembler 205 may be skipped via switches 205 a and 205 bsuch that decoded prediction partitions may have been generated atadaptive inverse transform module 204). The decoded predictionpartitions of prediction error data (e.g., prediction partitionresiduals) may be added to predicted partitions (e.g., prediction pixeldata) at adder 206 to generate reconstructed prediction partitions. Thereconstructed prediction partitions may be transmitted to predictionpartitions assembler 207. Prediction partitions assembler 207 mayassemble the reconstructed prediction partitions to generatereconstructed tiles or super-fragments. In some examples, codingpartitions assembler module 205 and prediction partitions assemblermodule 207 may together be considered an un-partitioner subsystem ofdecoder 200.

The reconstructed tiles or super-fragments may be transmitted to deblockfiltering module 208. Deblock filtering module 208 may deblock anddither the reconstructed tiles or super-fragments (or predictionpartitions of tiles or super-fragments). The generated deblock anddither filter parameters may be determined from input bitstream 201, forexample. The output of deblock filtering module 208 may be transmittedto a quality restoration filtering module 209. Quality restorationfiltering module 209 may apply quality filtering based on QR parameters,which may be determined from input bitstream 201, for example. As shownin FIG. 2, the output of quality restoration filtering module 209 may betransmitted to decoded picture buffer 210. In some examples, the outputof quality restoration filtering module 209 may be a final reconstructedframe that may be used for prediction for coding other frames (e.g., thefinal reconstructed frame may be a reference frame or the like). In someexamples, deblock filtering module 208 and quality restoration filteringmodule 209 may together be considered a filtering subsystem of decoder200.

As discussed, compensation due to prediction operations may includeinter- and/or intra-prediction compensation. As shown, inter-predictioncompensation may be performed by one or more modules including morphinggeneration module 211, synthesizing generation module 212, andcharacteristics and motion compensated filtering predictor module 213.Morphing generation module 211 may use de-quantized morphing parameters(e.g., determined from input bitstream 201) to generate morphedreference frames. Synthesizing generation module 212 may generate superresolution (SR) pictures and projected interpolation (PI) pictures orthe like based on parameters determined from input bitstream 201. Ifinter-prediction is applied, characteristics and motion compensatedfiltering predictor module 213 may apply motion compensation based onthe received frames and motion vector data or the like in inputbitstream 201.

Intra-prediction compensation may be performed by intra-directionalprediction generation module 214. Intra-directional predictiongeneration module 214 may be configured to perform spatial directionalprediction and may use decoded neighboring partitions according tointra-prediction data in input bitstream 201.

As shown in FIG. 2, prediction modes selector module 215 may determine aprediction mode selection from among, “skip”, “auto”, “inter”, “multi”,and “intra”, for each prediction partition of a tile, all of which mayapply to P- and F/B-pictures, based on mode selection data in inputbitstream 201. In addition to prediction modes, it also allows forselection of reference types that can be different depending on “inter”or “multi” mode, as well as for P- and F/B-pictures. The predictionsignal at the output of prediction modes selector module 215 may befiltered by prediction fusion filtering module 216. Prediction fusionfiltering module 216 may perform filtering based on parameters (e.g.,filtering coefficients, frequency, overhead) determined via inputbitstream 201. In some examples, filtering the prediction signal mayfuse different types of signals representing different modes (e.g.,intra, inter, multi, skip, and auto). In some examples, intra-predictionsignals may be different than all other types of inter-predictionsignal(s) such that proper filtering may greatly enhance codingefficiency. The filtered prediction signal may provide the second input(e.g., prediction partition(s)) to differencer 206, as discussed above.

As discussed, the output of quality restoration filtering module 209 maybe a final reconstructed frame. Final reconstructed frames may betransmitted to an adaptive picture re-organizer 217, which may re-orderor re-organize frames as needed based on ordering parameters in inputbitstream 201. Re-ordered frames may be transmitted to contentpost-restorer module 218. Content post-restorer module 218 may be anoptional module configured to perform further improvement of perceptualquality of the decoded video. The improvement processing may beperformed in response to quality improvement parameters in inputbitstream 201 or it may be performed as standalone operation. In someexamples, content post-restorer module 218 may apply parameters toimprove quality such as, for example, an estimation of film grain noiseor residual blockiness reduction (e.g., even after the deblockingoperations discussed with respect to deblock filtering module 208). Asshown, decoder 200 may provide display video 219, which may beconfigured for display via a display device (not shown).

FIG. 2 illustrates example control signals associated with operation ofvideo decoder 200, where the indicated abbreviations may representsimilar information as discussed with respect to FIG. 1 above. Whilethese control signals are illustrated as being associated with specificexample functional modules of decoder 200 in FIG. 4, otherimplementations may include a different distribution of control signalsamong the functional modules of encoder 100. The present disclosure isnot limited in this regard and, in various examples, implementation ofthe control signals herein may include the undertaking of only a subsetof the specific example control signals shown, additional controlsignals, and/or in a different arrangement than illustrated.

While FIGS. 1 through 2 illustrate particular encoding and decodingmodules, various other coding modules or components not depicted mayalso be utilized in accordance with the present disclosure. Further, thepresent disclosure is not limited to the particular componentsillustrated in FIGS. 1 and 2 and/or to the manner in which the variouscomponents are arranged. Various components of the systems describedherein may be implemented in software, firmware, and/or hardware and/orany combination thereof. For example, various components of encoder 100and/or decoder 200 may be provided, at least in part, by hardware of acomputing System-on-a-Chip (SoC) such as may be found in a computingsystem such as, for example, a mobile phone.

Further, it may be recognized that encoder 100 may be associated withand/or provided by a content provider system including, for example, avideo content server system, and that output bitstream 111 may betransmitted or conveyed to decoders such as, for example, decoder 200 byvarious communications components and/or systems such as transceivers,antennae, network systems, and the like not depicted in FIGS. 1 and 2.It may also be recognized that decoder 200 may be associated with aclient system such as a computing device (e.g., a desktop computer,laptop computer, tablet computer, convertible laptop, mobile phone, orthe like) that is remote to encoder 100 and that receives inputbitstream 201 via various communications components and/or systems suchas transceivers, antennae, network systems, and the like not depicted inFIGS. 1 and 2. Therefore, in various implementations, encoder 100 anddecoder subsystem 200 may be implemented either together or independentof one another.

FIG. 3(a) is an illustrative diagram of an example next generation videoencoder 300 a, arranged in accordance with at least some implementationsof the present disclosure. FIG. 3(a) presents a similar encoder to thatshown in FIGS. 1(a) and 1(b), and similar elements will not be repeatedfor the sake of brevity. As shown in FIG. 3(a), encoder 300 a mayinclude preanalyzer subsystem 310 a, partitioner subsystem 320 a,prediction encoding subsystem 330 a, transform encoder subsystem 340 a,filtering encoding subsystem 350 a, entropy encoder system 360 a,transform decoder subsystem 370 a, and/or unpartioner subsystem 380 a.Preanalyzer subsystem 310 a may include content pre-analyzer module 102and/or adaptive picture organizer module 104. Partitioner subsystem 320a may include prediction partitions generator module 105, and/or codingpartitions generator 107. Prediction encoding subsystem 330 a mayinclude motion estimator module 122, characteristics and motioncompensated filtering predictor module 123, and/or intra-directionalprediction analyzer and prediction generation module 124. Transformencoder subsystem 340 a may include adaptive transform module 108 and/oradaptive quantize module 109. Filtering encoding subsystem 350 a mayinclude blockiness analyzer and deblock filtering module 117, qualityanalyzer and quality restoration filtering module 118, motion estimatormodule 122, characteristics and motion compensated filtering predictormodule 123, and/or prediction analyzer and prediction fusion filteringmodule 126. Entropy coding subsystem 360 a may include adaptive entropyencoder module 110. Transform decoder subsystem 370 a may includeadaptive inverse quantize module 112 and/or adaptive inverse transformmodule 113. Unpartioner subsystem 380 a may include coding partitionsassembler 114 and/or prediction partitions assembler 116.

Partitioner subsystem 320 a of encoder 300 a may include twopartitioning subsystems: prediction partitions generator module 105 thatmay perform analysis and partitioning for prediction, and codingpartitions generator module 107 that may perform analysis andpartitioning for coding. Another partitioning method may includeadaptive picture organizer 104 which may segment pictures into regionsor slices may also be optionally considered as being part of thispartitioner.

Prediction encoder subsystem 330 a of encoder 300 a may include motionestimator 122 and characteristics and motion compensated filteringpredictor 123 that may perform analysis and prediction of “inter”signal, and intra-directional prediction analyzer and predictiongeneration module 124 that may perform analysis and prediction of“intra” signal. Motion estimator 122 and characteristics and motioncompensated filtering predictor 123 may allow for increasingpredictability by first compensating for other sources of differences(such as gain, global motion, registration), followed by actual motioncompensation. They may also allow for use of data modeling to createsynthesized frames (super resolution, and projection) that may allowbetter predictions, followed by use of actual motion compensation insuch frames.

Transform encoder subsystem 340 a of encoder 300 a may perform analysisto select the type and size of transform and may include two major typesof components. The first type of component may allow for usingparametric transform to allow locally optimal transform coding of smallto medium size blocks; such coding however may require some overhead.The second type of component may allow globally stable, low overheadcoding using a generic/fixed transform such as the DCT, or a picturebased transform from a choice of small number of transforms includingparametric transforms. For locally adaptive transform coding, PHT(Parametric Haar Transform) may be used. Transforms may be performed on2D blocks of rectangular sizes between 4×4 and 64×64, with actual sizesthat may depend on a number of factors such as if the transformed datais luma or chroma, inter or intra, and if the transform used is PHT orDCT. The resulting transform coefficients may be quantized, scanned andentropy coded.

Entropy encoder subsystem 360 a of encoder 300 a may include a number ofefficient but low complexity components each with the goal ofefficiently coding a specific type of data (various types of overhead,motion vectors, or transform coefficients). Components of this subsystemmay belong to a generic class of low complexity variable length codingtechniques, however, for efficient coding, each component may be customoptimized for highest efficiency. For instance, a custom solution may bedesigned for coding of “Coded/Not Coded” data, another for “Modes andRef Types” data, yet another for “Motion Vector” data, and yet anotherone for “Prediction and Coding Partitions” data. Finally, because a verylarge portion of data to be entropy coded is “transform coefficient”data, multiple approaches for efficient handling of specific blocksizes, as well as an algorithm that may adapt between multiple tablesmay be used.

Filtering encoder subsystem 350 a of encoder 300 a may perform analysisof parameters as well as multiple filtering of the reconstructedpictures based on these parameters, and may include several subsystems.For example, a first subsystem, blockiness analyzer and deblockfiltering module 117 may deblock and dither to reduce or mask anypotential block coding artifacts. A second example subsystem, qualityanalyzer and quality restoration filtering module 118, may performgeneral quality restoration to reduce the artifacts due to quantizationoperation in any video coding. A third example subsystem, which mayinclude motion estimator 122 and characteristics and motion compensatedfiltering predictor module 123, may improve results from motioncompensation by using a filter that adapts to the motion characteristics(motion speed/degree of blurriness) of the content. A fourth examplesubsystem, prediction fusion analyzer and filter generation module 126,may allow adaptive filtering of the prediction signal (which may reducespurious artifacts in prediction, often from intra prediction) therebyreducing the prediction error which needs to be coded.

Encode controller module 103 of encoder 300 a may be responsible foroverall video quality under the constraints of given resources anddesired encoding speed. For instance, in full RDO (Rate DistortionOptimization) based coding without using any shortcuts, the encodingspeed for software encoding may be simply a consequence of computingresources (speed of processor, number of processors, hyperthreading,DDR3 memory etc.) availability. In such case, encode controller module103 may be input every single combination of prediction partitions andcoding partitions and by actual encoding, and the bitrate may becalculated along with reconstructed error for each case and, based onlagrangian optimization equations, the best set of prediction and codingpartitions may be sent for each tile of each frame being coded. The fullRDO based mode may result in best compression efficiency and may also bethe slowest encoding mode. By using content analysis parameters fromcontent preanalyzer module 102 and using them to make RDO simplification(not test all possible cases) or only pass a certain percentage of theblocks through full RDO, quality versus speed tradeoffs may be madeallowing speedier encoding. Up to now we have described a variablebitrate (VBR) based encoder operation. Encode controller module 103 mayalso include a rate controller that can be invoked in case of constantbitrate (CBR) controlled coding.

Lastly, preanalyzer subsystem 310 a of encoder 300 a may performanalysis of content to compute various types of parameters useful forimproving video coding efficiency and speed performance. For instance,it may compute horizontal and vertical gradient information (Rs, Cs),variance, spatial complexity per picture, temporal complexity perpicture, scene change detection, motion range estimation, gaindetection, prediction distance estimation, number of objects estimation,region boundary detection, spatial complexity map computation, focusestimation, film grain estimation etc. The parameters generated bypreanalyzer subsystem 310 a may either be consumed by the encoder or bequantized and communicated to decoder 200.

While subsystems 310 a through 380 a are illustrated as being associatedwith specific example functional modules of encoder 300 a in FIG. 3(a),other implementations of encoder 300 a herein may include a differentdistribution of the functional modules of encoder 300 a among subsystems310 a through 380 a. The present disclosure is not limited in thisregard and, in various examples, implementation of the examplesubsystems 310 a through 380 a herein may include the undertaking ofonly a subset of the specific example functional modules of encoder 300a shown, additional functional modules, and/or in a differentarrangement than illustrated.

FIG. 3(b) is an illustrative diagram of an example next generation videodecoder 300 b, arranged in accordance with at least some implementationsof the present disclosure. FIG. 3(b) presents a similar decoder to thatshown in FIG. 2, and similar elements will not be repeated for the sakeof brevity. As shown in FIG. 3(b), decoder 300 b may include predictiondecoder subsystem 330 b, filtering decoder subsystem 350 b, entropydecoder subsystem 360 b, transform decoder subsystem 370 b,unpartitioner 2 subsystem 380 b, unpartitioner 1 subsystem 351 b,filtering decoder subsystem 350 b, and/or postrestorer subsystem 390 b.Prediction decoder subsystem 330 b may include characteristics andmotion compensated filtering predictor module 213 and/orintra-directional prediction generation module 214. Filtering decodersubsystem 350 b may include deblock filtering module 208, qualityrestoration filtering module 209, characteristics and motion compensatedfiltering predictor module 213, and/or prediction fusion filteringmodule 216. Entropy decoder subsystem 360 b may include adaptive entropydecoder module 202. Transform decoder subsystem 370 b may includeadaptive inverse quantize module 203 and/or adaptive inverse transformmodule 204. Unpartitioner_2 subsystem 380 b may include codingpartitions assembler 205. Unpartitioner_1 subsystem 351 b may includeprediction partitions assembler 207. Postrestorer subsystem 790 mayinclude content post restorer module 218 and/or adaptive picturere-organizer 217.

Entropy decoding subsystem 360 b of decoder 300 b may perform theinverse operation of the entropy encoder subsystem 360 a of encoder 300a, i.e., it may decode various data (types of overhead, motion vectors,transform coefficients) encoded by entropy encoder subsystem 360 a usinga class of techniques loosely referred to as variable length decoding.Specifically, various types of data to be decoded may include “Coded/NotCoded” data, “Modes and Ref Types” data, “Motion Vector” data,“Prediction and Coding Partitions” data, and “Transform Coefficient”data.

Transform decoder subsystem 370 b of decoder 300 b may perform inverseoperation to that of transform encoder subsystem 340a of encoder 300 a.Transform decoder subsystem 370 b may include two types of components.The first type of example component may support use of the parametricinverse PHT transform of small to medium block sizes, while the othertype of example component may support inverse DCT transform for allblock sizes. The PHT transform used for a block may depend on analysisof decoded data of the neighboring blocks. Output bitstream 111 and/orinput bitstream 201 may carry information about partition/block sizesfor PHT transform as well as in which direction of the 2D block to beinverse transformed the PHT may be used (the other direction uses DCT).For blocks coded purely by DCT, the partition/block sizes informationmay be also retrieved from output bitstream 111 and/or input bitstream201and used to apply inverse DCT of appropriate size.

Unpartitioner subsystem 380 b of decoder 300 b may perform inverseoperation to that of partitioner subsystem 320 a of encoder 300 a andmay include two unpartitioning subsystems, coding partitions assemblermodule 205 that may perform unpartitioning of coded data and predictionpartitions assembler module 207 that may perform unpartitioning forprediction. Further if optional adaptive picture organizer module 104 isused at encoder 300 a for region segmentation or slices, adaptivepicture re-organizer module 217 may be needed at the decoder.

Prediction decoder subsystem 330 b of decoder 300 b may includecharacteristics and motion compensated filtering predictor module 213that may perform prediction of “inter” signal and intra-directionalprediction generation module 214 that may perform prediction of “intra”signal. Characteristics and motion compensated filtering predictormodule 213 may allow for increasing predictability by first compensatingfor other sources of differences (such as gain, global motion,registration) or creation of synthesized frames (super resolution, andprojection), followed by actual motion compensation.

Filtering decoder subsystem 350 b of decoder 300 b may perform multiplefiltering of the reconstructed pictures based on parameters sent byencoder 300 a and may include several subsystems. The first examplesubsystem, deblock filtering modue 208, may deblock and dither to reduceor mask any potential block coding artifacts. The second examplesubsystem, quality restoration filtering module 209, may perform generalquality restoration to reduce the artifacts due to quantizationoperation in any video coding. The third example subsystem,characteristics and motion compensated filtering predictor module 213,may improve results from motion compensation by using a filter that mayadapt to the motion characteristics (motion speed/degree of blurriness)of the content. The fourth example subsystem, prediction fusionfiltering module 216, may allow adaptive filtering of the predictionsignal (which may reduce spurious artifacts in prediction, often fromintra prediction) thereby reducing the prediction error which may needto be coded.

Postrestorer subsystem 390 b of decoder 300 b is an optional block thatmay perform further improvement of perceptual quality of decoded video.This processing can be done either in response to quality improvementparameters sent by encoder 100, or it can be standalone decision made atthe postrestorer subsystem 390 b. In terms of specific parameterscomputed at encoder 100 that can be used to improve quality atpostrestorer subsystem 390 b may be estimation of film grain noise andresidual blockiness at encoder 100 (even after deblocking). As regardsthe film grain noise, if parameters can be computed and sent via outputbitstream 111 and/or input bitstream 201to decoder 200, then theseparameters may be used to synthesize the film grain noise. Likewise, forany residual blocking artifacts at encoder 100, if they can be measuredand parameters sent via output bitstream 111 and/or bitstream 201,postrestorer subsystem 390 b may decode these parameters and may usethem to optionally perform additional deblocking prior to display. Inaddition, encoder 100 also may have access to scene change, spatialcomplexity, temporal complexity, motion range, and prediction distanceinformation that may help in quality restoration in postrestorersubsystem 390 b.

While subsystems 330 b through 390 b are illustrated as being associatedwith specific example functional modules of decoder 300 b in FIG. 3(b),other implementations of decoder 300 b herein may include a differentdistribution of the functional modules of decoder 300 b among subsystems330 b through 390 b. The present disclosure is not limited in thisregard and, in various examples, implementation of the examplesubsystems 330 b through 390 b herein may include the undertaking ofonly a subset of the specific example functional modules of decoder 300b shown, additional functional modules, and/or in a differentarrangement than illustrated.

FIG. 4 is an illustrative diagram of an example entropy encoder module110, arranged in accordance with at least some implementations of thepresent disclosure. As shown, entropy encoder module 110 may includebitstream headers, parameters and maps data encoder module 401, picturepartitions, prediction partitions, and coding partitions encoder module402, coding modes and reference types encoder module 403,coded/not-coded data encoder module 404, transform coefficients encodermodule 405, motion vector prediction and differential motion vectorencoder module 406, intra-prediction type and direction data encodermodule 407, and/or bitstream assembler module 408. In the discussionherein, each of modules 401-407 may be shortened to encoder module 401,encoder module 404, or the like for the sake of brevity.

As shown, encoder modules 401-407 may receive video data 411-417,respectively, via adaptive entropy encoder 110. In some examples,received video data 411-417 may be received from various modules ofencoder 100 as discussed herein. As shown, encoder modules 401-407 maycompress the received video data 411-417 to generated compressed videodata 421-427. Compressed video data 421-427 may be provided to bitstreamassembler 408, which may assemble compressed video data 421-427 togenerate output bitstream 111.

In some examples, encoder modules 401-407 may each include one or morespecialized component(s) for efficiently encoding the type of dataassociated with received video data 411-417. In some examples, one ormore of encoder modules 401-407 may preprocess the received video data411-417 and/or select an entropy coding technique based on a parameter,parameters, or characteristics of the received video data 411-417 orother system information.

For example, encoder module 401 may receive overhead data 411, which mayinclude bitstream header data (e.g., sequence and/or picture levelbitstream headers), morphing parameters, synthesizing parameters, orglobal maps data (e.g., quantizer maps of pictures indicating quantizersto be used on a partition basis). As is discussed further below withrespect to FIG. 6, in some examples, encoder module 401 may implement anadaptive symbol-run variable length coding technique, an adaptive proxyvariable length coding technique, or a variable length coding table ortables compression of video data 411. In some examples, encoder module411 may determine which technique provides the greatest compressionefficiency (e.g., the fewest bits for compressed video data 421) suchthat the parameter(s) associated with video data 411 may be the numberof bits needed for each coding technique or the like. Encoder module 411may entropy encode video data 411 to generate compressed video data 421(e.g., compressed overhead data), which may be transmitted to bitstreamassembler 408 as shown.

As discussed, in some examples, the parameter associated with the videodata (e.g., any of video data 411-417) may be a fewest number ofattainable bits, most efficient encoding technique, or the like. Inother examples, the parameter associated with the video data may be apredefined or predetermine parameter such that encoding technique ispredetermined. In some examples, the parameter associated with the videodata may be based on a characteristic of the video data such that thedetermined encoding technique may be adaptive to the received video dataas is discussed further herein.

As shown, in some examples, encoder module 402 may receive partitiondata 412, which may include picture slices or regions data,intra-prediction partition data, and/or inter-prediction partition andcoding partition data. As is discussed further below with respect toFIG. 6, in some examples, encoder module 412 may implement an adaptivesymbol-run variable length coding technique or an adaptive proxyvariable length coding technique for the compression of theintra-prediction partition data and/or inter-prediction partition dataportions of video data 412 based on a parameter associated with theintra-prediction partition data and/or inter-prediction partition data(e.g., a fewest number of attainable bits, most efficient encodingtechnique, a predetermined parameter, characteristics of video data 412,or the like), and encoder module 412 may implement adaptive codebookvariable length coding for the slices or regions data portions of videodata 412 to generate compressed video data 422 (e.g., compressedpartition data), which may be transmitted to bitstream assembler 408 asshown. In some examples, the intra-prediction partition data and/orinter-prediction partition data may include data indicating thepartitioning of tiles into partitions, partitions into sub-partitions,or the like. In some examples, the partitions and/or sub-partitions mayinclude prediction partitions and/or sub-partitions. In some examples,partitions and/or sub-partitions may include coding partitions and/orsub-partitions.

Further as shown, in some examples, encoder module 403 may receive modesand reference types data 413, which may include modes (e.g., intra,split, skip, auto, inter, or multi) data and/or references data for eachprediction partition. For example, the mode split information mayindicate whether a partition is further divided or not. If a partitionis further divided, the mode data may further include directioninformation indicating whether the split is a horizontal split (e.g.,hor) or a vertical split (e.g., vert). As is discussed further belowwith respect to FIG. 6, in some examples, encoder module 403 mayimplement an adaptive symbol-run variable length coding technique or anadaptive proxy variable length coding technique for separate coding ofsplit/non-split partition information data, separate coding ofsplit/non-split split information data, or prediction referenceinformation data based on a parameter associated with the data (e.g., afewest number of attainable bits, most efficient encoding technique, apredetermined parameter, characteristics of the pertinent portions ofvideo data 413, or the like), and encoder module 403 may implementadaptive variable length coding for joint coding of modes and splitinformation to generate compressed video data 423(e.g., compressed modesand reference types data), which may be transmitted to bitstreamassembler 408 as shown.

Further, in some examples, encoder module 404 may receivecoded/not-coded data 414, which may include coded/not-coded data asdiscussed herein. For example, a partition (or sub-partition) may becoded if it has any nonzero transform coefficients and a partition (orsub-partition) may be not-coded if it has all zero transformcoefficients. In some examples, coded/not-coded data may not be neededfor partitions having an intra or skip mode. In some examples,coded/not-coded data may be needed for partitions having an auto, inter,or multi mode. As is discussed further below with respect to FIG. 6, insome examples, encoder module 404 may implement an adaptive symbol-runvariable length coding technique or an adaptive proxy variable lengthcoding technique for coded/not-coded data based on a parameterassociated with the coded/not-coded data (e.g., a fewest number ofattainable bits, most efficient encoding technique, a predeterminedparameter, characteristics of video data 414, or the like) to generatecompressed video data 424 (e.g., compressed coded/not-coded data), whichmay be transmitted to bitstream assembler 408 as shown.

In some examples, encoder module 405 may receive transform data 415,which may include transform coefficient data. For example, for blocks orpartitions or sub-partitions that are coded (e.g., have one or morenonzero transform coefficients), transform coefficient data may bereceived for entropy encoding. As is discussed further with respect toFIG. 6, encoder module 405 may implement adaptive vector variable lengthcoding for blocks or partitions or sub-partitions having a block orpartition or sub-partition size of 2 in one dimension (e.g., 2×Kpartitions or K×2 sized partitions). Further, encoder module 405 mayimplement adaptive 1-dimensional variable length coding for blocks orpartitions or sub-partitions of size 4×4 and adaptive 2-dimensionalvariable length coding for all other block or partition or sub-partitionsizes (e.g., 4×8, 8×4, 8×8, 16×16, 32×32, 64×64, and so on). Thegenerated compressed video data 425 (e.g., compressed transform data)may be transmitted to bitstream assembler 408 as shown.

For example, the transform coefficient data may result from a forwardtransform of rectangular (or square or the like) partitions of pixeldata or rectangular (or square or the like) of pixel difference valuesimplemented via adaptive transform module 108 followed by a quantizationof the resulting coefficients via adaptive quantize module 109. In someexamples, the transform coefficient data may be scanned to convert it toa 1-dimensional frequency ordered partition via encoder module 405. Suchconversion may be highly adaptive any partition size (e.g., 24 or moreor 32 or more partition sizes), different data types (e.g., discretecosine transform coefficients or hybrid parametric Haar transformcoefficients or the like of either intra or inter partitions), and/ordifferent quantizer set characteristics (e.g., various combinations ofquantizer parameters and/or matrices). Further, a block or partition orsub-partition may belong to different picture types: I-picture (e.g.,intra compensation only), P-picture (e.g., predictive) or F-picture(e.g., functional) and/or may represent different types of signal ordata (e.g., luma or chroma or the like), which may be quantized withdifferent quantizer setting.

Further, in some examples, encoder module 406 may receive motion data416, which may include motion vector data. As is discussed further withrespect to FIG. 6, motion vector prediction may be performed based onvideo data 416 to generate one or more predicted motion vectors. Apredicted motion vector may be differenced with an original motion dataof video data 416 to generate a difference motion vector. In someexamples, encoder module 416 may implement an adaptive classifiedvariable length coding for the difference motion vector(s) to generatecompressed video data 426 (e.g., compressed motion data), which may betransmitted to bitstream assembler 408 as shown.

Further, in some examples, encoder module 407 may receiveintra-prediction data 417, which may include intra-prediction type orintra-prediction direction data. For example, as discussed, intra codingmay use prediction, which may use neighboring past decoded partition(s)within the same frame to generate spatial prediction. In such examples,there may be predictors for indicating a past decoded partition orpartitions. For example, the predictors may include dc, slope,directional, BTPC, feature matching, or the like. Further, in someexamples, the directional predictor may be adaptive for differentpartition sizes. For example, specifying a directional predictor mayinclude providing an technique for determining angular prediction pixelpartition(s) for coding using causal neighboring decoded partitionsand/or specifying a technique for entropy coding spatial predictiondirections. In some examples, such techniques may be performed viaencoder module 407. As is discussed further below with respect to FIG.6, in some examples, encoder module 407 may implement an adaptivevariable length coding technique or an arithmetic coding technique forintra-prediction type or intra-prediction direction data based on aparameter associated with the intra-prediction type or intra-predictiondirection data (e.g., a fewest number of attainable bits, most efficientencoding technique, a predetermined parameter, characteristics of videodata 417, or the like) to generate compressed video data 427 (e.g.,compressed intra-prediction data), which may be transmitted to bitstreamassembler 408 as shown.

As shown in FIG. 4, adaptive entropy encoder 110 may include bitstreamassembler 408. In some examples, some or all of encoder modules 401-407may provide entropy coded compressed video data 421-427 at differentinstances in time. Further, in some examples, one or some of compressedvideo data 421-427 may be picture based, region or slice based, tilebased, prediction partition based, coding partition based, or anycombination thereof. In some examples, bitstream assembler may multiplex(the potentially different) compressed video data 421-427 to create avalid bitstream such as, for example, output bitstream 111. For example,the valid bitstream may be a valid next generation video (NGV) codedbitstream, which may following a NGV bitstream syntax specification. Insome examples, output bitstream 111 may be a video only bitstream. Insome examples, output bitstream 111 may be multiplexed (e.g., Transportor a Media File Format) with uncoded or coded audio to create amultiplexed audio-visual stream. In any event, the bitstream may be usedlocal decode, storage, or transmission to a decoder as discussed herein.

FIG. 5 is an illustrative diagram of an example entropy decoder module202, arranged in accordance with at least some implementations of thepresent disclosure. As shown, entropy decoder module 202 may includebitstream headers, parameters and maps data decoder module 501, picturepartitions, prediction partitions, and coding partitions decoder module502, coding modes and reference types decoder module 503,coded/not-coded data decoder module 504, transform coefficients decodermodule 505, motion vector and differential motion vector decoder module506, intra-prediction and direction data decoder module 507, and/orbitstream disassembler module 508. In the discussion herein, each ofmodules 501-507 may be shortened to decoder module 501, decoder module505, or the like for the sake of brevity.

As shown, bitstream disassembler module 508 may receive input bitstream201. In some examples, input bitstream 201 may be a valid bitstream suchas, for example, a valid next generation video (NGV) coded bitstream,which may follow a NGV bitstream syntax specification. In some examples,input bitstream 201 may be a video only bitstream. In some examples,input bitstream 201 may be a multiplexed audio-visual stream asdiscussed herein. Bitstream disassembler module 508 may disassembleinput bitstream 201 to determine compressed video data 511-517 as shown.For example, bitstream disassembler module 508 may use a predefinedsyntax or specification to divide input bitstream 201 into componentcompressed video data 511-517 by data type for decompression via decodermodules 501-507. In some examples, bitstream disassembler module 508 mayperform an inverse operation with respect to bitstream assembler module508.

As shown in FIG. 5, decoder modules 501-507 may receive compressed videodata 511-517, respectively, and generate video data 521-527. Video data521-527 may be transmitted to various components of decoder 200 forfurther decoding as discussed herein. Decoder 200 may thereby generatevideo frame(s) for presentment to a user via a display device (notshown). In some examples, decoder modules 501-507 may each perform aninverse operation with respect to encoder modules 401-407. In someexamples, decoder modules 501-507 may each include one or morespecialized component(s) for efficiently entropy decoding the type ofdata associated with compressed video data 511-517.

For example, decoder module 501 may receive compressed overhead data511, which may include compressed bitstream header data (e.g., sequenceand/or picture level bitstream headers), morphing parameters,synthesizing parameters, or global maps data (e.g., quantizer maps ofpictures indicating quantizers to be used on a partition basis). In someexamples, decoder module 511 may implement an adaptive symbol-runvariable length coding technique, an adaptive proxy variable lengthcoding technique, or a variable length coding table or tables fordecompression of compressed overhead data 511 to generate overhead data521. In some examples, decoder module 501 may determine which codingtechnique to implement based on a parameter or indicator provided viabitstream 201.

As shown, in some examples, decoder module 502 may receive compressedpartition data 512, which may include compressed picture slices orregions data, intra-prediction partition data, and/or inter-predictionpartition data. In some examples, decoder module 512 may implement anadaptive symbol-run variable length coding technique or an adaptiveproxy variable length coding technique for the decompression of theintra-prediction partition data and/or inter-prediction partition dataportions of compressed partition data 512, and decoder module 512 mayimplement an adaptive codebook variable length coding for thedecompression of the slices or regions data portions of compressedpartition data 512 to generate partition data 522. In some examples, theintra-prediction partition data and/or inter-prediction partition datamay include data indicating the partitioning of tiles into partitions,partitions into sub-partitions, or the like. In some examples, thepartitions and/or sub-partitions may include prediction partitionsand/or sub-partitions. In some examples, partitions and/orsub-partitions may include coding partitions and/or sub-partitions. Insome examples, decoder module 502 may determine which coding techniqueto implement for the decompression of the intra-prediction partitiondata and/or inter-prediction partition data portions of compressed videodata 512 based on a parameter or indicator provided via bitstream 201.

Further, in some examples, decoder module 503 may receive compressedmodes and reference types data 513, which may include compressed modes(e.g., intra, split, skip, auto, inter, or multi) data and/or referencesdata for each prediction partition. For example, the mode splitinformation may indicate whether a partition is further divided or not.If a partition is further divided, the mode data may further includedirection information indicating whether the split is a horizontal split(e.g., hor) or a vertical split (e.g., vert). In some examples, decodermodule 503 may implement an adaptive symbol-run variable length codingtechnique or an adaptive proxy variable length coding technique fordecompression of separate coding of split/non-split partitioninformation data, separate coding of split/non-split split informationdata, or prediction reference information data, and decoder module 503may implement an adaptive variable length coding for decompression ofjoint coding of modes and split information to generate modes andreference types data 523. In some examples, decoder module 503 maydetermine which coding technique to implement for decompression ofseparate coding of split/non-split partition information data, separatecoding of split/non-split split information data, or predictionreference information data based on a parameter or indicator providedvia bitstream 201.

Further, in some examples, decoder module 504 may receive compressedcoded/not-coded data 514, which may include coded/not-coded data asdiscussed herein. For example, a partition (or sub-partition) may becoded if it has any nonzero transform coefficients and a partition (orsub-partition) may be not-coded if it has all zero transformcoefficients. In some examples, coded/not-coded data may not be neededfor partitions having an intra or skip mode. In some examples,coded/not-coded data may be needed for partitions having an auto, inter,or multi mode. In some examples, decoder module 504 may implement anadaptive symbol-run variable length coding technique or an adaptiveproxy variable length coding technique for decompression ofcoded/not-coded data to generate coded/not-coded data 524. In someexamples, decoder module 504 may determine which coding technique toimplement for decompression based on a parameter or indicator providedvia bitstream 201.

As shown, in some examples, decoder module 505 may receive compressedtransform data 515, which may include transform coefficient data. Forexample, for blocks or partitions or sub-partitions that are coded(e.g., have one or more nonzero transform coefficients) compressedtransform data 515 may include transform coefficient data. In someexamples, decoder module 505 may implement an adaptive vector variablelength coding for decompression of blocks or partitions orsub-partitions having a block or partition or sub-partition size of 2 inone dimension (e.g., 2×K partitions or K×2 sized partitions. Further,decoder module 505 may implement an adaptive 1-dimensional variablelength coding for decompression of blocks or partitions orsub-partitions of size 4×4 and adaptive 2-dimensional variable lengthcoding for decompression of all other block or partition orsub-partition sizes (e.g., 4×8, 8×4, 8×8, 16×16, 32×32, 64×64, and soon). The generated transform data 525 may be transmitted to othermodule(s) of decoder 200 as shown.

Further, in some examples, decoder module 506 may receive compressedmotion data 516, which may include motion vector data. As is discussedfurther below with respect to FIG. 6, in some examples, compressedmotion data 516 may be decompressed using an adaptive classifiedvariable length coding technique to generate predicted difference motionvectors. The predicted difference motion vectors may be added to motionvector prediction to generate reconstructed motion vectors. The motionvector prediction may be generated based on previously decoded motionvectors of neighboring blocks or partitions using the inverse of thetechnique implemented via encoder module 506, for example, and/or motionvectors. The reconstructed motion vectors may be transmitted to othermodule(s) of decoder 200 via motion data 526 as shown.

Further, in some examples, decoder module 507 may receive compressedintra-prediction data 517, which may include intra-prediction type orintra-prediction direction data. For example, as discussed, intra codingmay use prediction, which may use neighboring past decoded partition(s)within the same frame to generate spatial prediction. In such examples,there may be predictors for indicating a past decoded partition orpartitions. For example, the predictors may include dc, slope,directional, BTPC, feature matching, or the like. Further, in someexamples, the directional predictor may be adaptive for differentpartition sizes. For example, specifying a directional predictor mayinclude providing an technique for determining angular prediction pixelpartition(s) for coding using causal neighboring decoded partitionsand/or specifying a technique for entropy coding spatial predictiondirections. In some examples, decoder module 517 may implement anadaptive variable length coding technique or an arithmetic codingtechnique for decompression of intra-prediction type or intra-predictiondirection data to generate intra-prediction data 527. In some examples,decoder module 507 may determine which coding technique to implement fordecompression based on a parameter or indicator provided via bitstream201.

As discussed above, a variety of entropy coding techniques may beimplemented on various data types for lossless compression of video datato generate compressed video data at an entropy encoder and fordecompression of the compressed video data to generate duplicate videodata at an entropy decoder.

In some examples, an adaptive symbol-run variable length codingtechnique may be implemented. For example, encoder and decoder modules401, 501, 402, 502, 403, 503, and/or 404, 504 may implement an adaptivesymbol-run variable length coding technique on some or all of the videodata or compressed video data received.

In some examples, an adaptive symbol-run variable length codingtechnique may include coding of relative difference in addresses betweennon-skipped blocks within a frame in video coding that allows one todetermine the number of consecutive skipped blocks. For example, in thecontext of coded/not-coded data as encoded and decoded via encoder anddecoder modules 504, 504, instead of sending one bit (e.g., bit-map) foreach block to signal coded/not-coded (e.g., skipped) blocks, encodermodule 504, for example, may encode a run of skipped blocks. In suchimplementations, the longer the run of skipped blocks, the moreefficiently the data may be compressed.

Further, several types of adaptivity may be added to adaptive symbol-runvariable length coding technique as described herein: adaptivity thatmay allow for use of multiple tables, adaptivity that may allow for useof either performing this type of coding on original bit map data,inverted bitmap, differential bitmap, or gradient predictive bitmap, orthe like. For example, the adaptive symbol-run variable length codingtechnique may include converting the first video data from bit map datato at least one of an inverted bitmap, a differential bit map, or agradient predictive bit map before applying adaptive symbol-run variablelength coding. For example, adaptive symbol-run variable length codingtechnique may be used to entropy encode substantially any type of event(e.g., symbol/run combination). Further, symbol/run events may be usedto code multi-level (e.g., 0, 1, 2, 3, etc.) or binary (e.g., 0,1)events. In examples where multi-level events are encoded, adaptivesymbol-run variable length coding technique may be applied a number ofconsecutive times, breaking a multi-level map into a number of binarysub-maps, with each previous sub-map, excluded from next level'ssub-map, or the like.

In some examples, an adaptive proxy variable length coding technique maybe implemented. For example, encoder and decoder modules 401, 501, 402,502, 403, 503, and/or 404, 504 may implement an adaptive proxy variablelength coding technique on some or all of the video data or compressedvideo data received.

In some examples, an adaptive proxy variable length coding technique mayinclude substitution of original fixed length 1D blocks (e.g., groups)of bits with variable length codes (e.g., patterns of sequence of bits)such that after the replacement the resulting bitstream may be smallerin size. In some examples, at the decoder, the process may be repeated(or inversed) resulting in original intended bitstream. In someexamples, the original blocks of bits replaced may be of fixed smallsizes (e.g., groups of 2 bits, groups of 3 bits, or groups of 4 bits, orthe like). In some examples, the replacement codes may be of small sizeand variable length in nature. In some examples, the adaptive proxyvariable length coding discussed herein may be characterized as ShortVLCs (e.g., variable length codes). Further, the adaptive proxy variablelength coding technique described herein may be adaptive to content byproviding multiple replacement options. In some examples, 1-dimensionalblocks/groups of 2 bits may be replaced with 1-3 bit long codes. In someexamples, 1-dimensional blocks/groups (or collections of blocks/groups)of 3 bits with codes may be replaced with 1-5 bit long codes. In someexamples, the adaptive proxy variable length coding technique mayexploit statistical redundancy within a bitstream. In some examples, theadaptive proxy variable length coding technique may provide acompression gain of about 1-1.3. In some examples, adaptive proxyvariable length coding technique may offer the advantage of beingamenable to application to short sequence of bits.

In some examples, an adaptive block-of-symbols variable length codingtechnique may be implemented. For example, encoder and decoder modules405, 505 may implement an adaptive block-of-symbols variable lengthcoding technique on some or all of the video data or compressed videodata received.

In some examples, an adaptive block-of-symbols variable length codingtechnique may include two sub-coding techniques, as will be discussedfurther with respect to FIG. 7. For example, the adaptiveblock-of-symbols variable length coding technique may include anadaptive vector variable length coding technique and an adaptive 1D/2D(1-dimensional/2-dimensional) variable length coding technique. In someexamples, the adaptive block-of-symbols variable length coding techniquemay be used to encode blocks of closely related symbols such as blocksof transform coefficients as discussed herein.

In some examples, the adaptive vector variable length coding techniqueof the adaptive block-of-symbols variable length coding technique mayencode relatively small 2D blocks or partitions of symbols by use of ajoint single codeword such that coding a block of symbols may result ina VLC (variable length coding) codebook. In some examples, the largerthe size of the block or partition, the larger the size of the codebook.In some examples, the adaptive vector variable length coding techniquemay be applied to block or partition sizes having a size 2 in onedimension (e.g., 2×K or K×2 blocks or partitions). By applying theadaptive vector variable length coding technique to blocks or partitionsof such sizes, the size of the VLC codebook may be advantageouslylimited.

In some examples, the adaptive 1D variable length coding technique ofthe adaptive block-of-symbols variable length coding technique may beused for coding 4×4 transform coefficient block or partition sizes. isessentially same as the CAVLC coder. This coder is primarily used forcoding 4×4. In some examples, the adaptive 1D variable length codingtechnique may be implemented via a content adaptive variable lengthcoding technique with a number of different VLC Tables used based on thecontext of the coefficient(s) being coded. For example, based on thecontext of the coefficient(s) being coded encoder and/or decoder modules505, 505 may switch VLC Tables.

In some examples, the adaptive 2D variable length coding technique ofthe adaptive block-of-symbols variable length coding technique mayutilize two dimensional properties of a block of symbols to switch basedon context between a number of different VCL Tables. In some examples,the adaptive 2D variable length coding technique may be characterized asa CA2DVLC (Content Adaptive 2D Variable Length) coder. In some examples,In some examples, the adaptive 2D variable length coding technique maybe used to encode all remaining transform coefficient block or petitionsizes besides 2×K, K×2 blocks and 4×4 blocks (e.g., 4×8, 8×4, 8×8,16×16, 32×32, 64×64, and so on).

FIG. 6 is an illustrative diagram of an example entropy encoder module110, arranged in accordance with at least some implementations of thepresent disclosure. As shown and as discussed above with respect to FIG.4, entropy encoder module 110 may include bitstream headers, parametersand maps data encoder module 401, picture partitions, predictionpartitions, and coding partitions encoder module 402, coding modes andreference types encoder module 403, coded/not-coded data encoder module404, transform coefficients encoder module 405, motion vector anddifferential motion vector encoder module 406, intra-prediction anddirection data encoder module 407, and/or bitstream assembler module408.

As shown, encoder module 401 may include adaptive VLC, symbol-run VLC,and/or proxy VLC encoder for bitstream headers, parameters, and mapsdata module 611 and may receive video data 411. Video data 411 may havea data type such that video data 411 may include bitstream header data(e.g., sequence and/or picture level bitstream headers), morphingparameters, synthesizing parameters, and/or global maps data (e.g.,quantizer maps of pictures indicating quantizers to be used on apartition basis). In some examples, an entropy encoding technique may bedetermined for video data 411 based on a parameter, parameters orcharacteristics of video data 411 or other system parameters. In someexamples, the entropy encoding technique for video data 411 may be oneof an adaptive symbol-run variable length coding technique or anadaptive proxy variable length coding technique, as described above, ora variable length coding table or tables compression technique. Thedetermined entropy encoding technique may be applied to video data 411to generate compressed video data 421. A variable length coding table ortables compression technique may include a content adaptive variablelength coding technique with one or more tables available for codingbased on video data 411, for example. In some examples, encoder module404 may determine which technique provides the greatest compressionefficiency (e.g., the fewest bits for compressed video data 421) suchthat the parameter(s) associated with video data 411 may be the numberof bits needed for each coding technique or the like. In some examples,the parameter associated with video data 411 may be based on acharacteristic of the video data such that the determined encodingtechnique may be adaptive to the received video data.

As shown in FIG. 6, encoder module 402 may include adaptive codebook VLCencoder for picture partitions module 621, symbol-run VLC and/or proxyVLC encoder for intra-prediction partitions module 622, and/orsymbol-run VLC and/or proxy VLC encoder for inter-prediction partitionsand coding partitions module 623. Also as shown, encoder module 402 mayreceive video data 412. In some examples, video data 412 may includepicture slices or regions data 624, intra-prediction partition data 625,and/or inter-prediction and coding partition data 626.

As shown, picture slices or regions data 624 may be received viaadaptive codebook VLC encoder for picture partitions module 621, whichmay apply adaptive codebook variable length coding to picture slices orregions data 624 to generate compressed picture slices or regions data627. In some examples, picture slices or regions data 624 may includeregion boundaries for pictures, slices, regions, or the like. In someexamples, adaptive codebook variable length coding may include contentadaptive variable length coding using an codebook adaptive to thecontent of picture slices or regions data 624 or other system parametersor the like.

As shown, intra-prediction partition data 625 may be received viasymbol-run VLC and/or proxy VLC encoder for intra-prediction partitionsmodule 622. In some examples, an entropy encoding technique may bedetermined for intra-prediction partition data 625 based on a parameter,parameters or characteristics of intra -prediction partition data 625 orother system parameters (e.g., compression efficiency, a characteristicof the data, and so on), as discussed herein. The determined entropyencoding technique may be applied to intra-prediction partition data 625to generate compressed intra-prediction partition data 628. As shown, insome examples, the entropy encoding technique for intra-predictionpartition data 625 may be one of an adaptive symbol-run variable lengthcoding technique or an adaptive proxy variable length coding technique,as described above. In some examples, intra-prediction partition data625 may include partitions based on bi-tree partitioning or k-d treepartitioning, or the like.

As shown, inter-prediction and coding partition data 626 may be receivedvia symbol-run VLC and/or proxy VLC encoder for inter-predictionpartitions and coding partitions module 623. In some examples, anentropy encoding technique may be determined for or inter-prediction andcoding partition data 626 based on a parameter, parameters orcharacteristics of or inter-prediction and coding partition data 626 orother system parameters (e.g., compression efficiency, a characteristicof the data, and so on), as discussed herein. The determined entropyencoding technique may be applied to or inter-prediction and codingpartition data 626 to generate compressed or inter-prediction and codingpartition data 629. As shown, in some examples, the entropy encodingtechnique for or inter-prediction and coding partition data 626 may beone of an adaptive symbol-run variable length coding technique or anadaptive proxy variable length coding technique, as described above. Insome examples, or inter-prediction and coding partition data 426 mayinclude inter-partitions and coding tree partitions, or the like.

As shown in FIG. 6, encoder module 403 may include adaptive VLC encoderfor joint modes and splits module 631, symbol-run VLC and/or proxy VLCencoder for modes module 632, symbol-run VLC and/or proxy VLC encoderfor splits module 633, and/or symbol-run VLC and/or proxy VLC encoderfor reference types module 634. Also as shown, encoder module 403 mayreceive video data 412. In some examples, video data 412 may jointcoding of modes and splits data 635, modes information data 636,split/not-split information data 637, and/or prediction referenceinformation data 638.

As shown, joint coding of modes and splits data 635 may be received viaadaptive VLC encoder for joint modes and splits module 631, which mayapply adaptive variable length coding to joint coding of modes andsplits data 635 to generate compressed joint coding of modes and splitsdata 639. In some examples, adaptive variable length coding may includecontent adaptive variable length coding adaptive to the content of jointcoding of modes and splits data 635 or other system parameters or thelike.

As discussed, in some examples, modes and splits data may be codedjointly via adaptive VLC encoder for joint modes and splits module 631.In some examples, modes and splits data may be coded separately viasymbol-run VLC and/or proxy VLC encoder for modes module 632 andsymbol-run VLC and/or proxy VLC encoder for splits module 633, as isdiscussed below. In some examples, encoder 100 (via, e.g., adaptiveentropy encoder 110 and/or encode controller 103) to code jointly orseparately based on results of a comparison of the coding techniques todetermine which technique compresses the data most efficiently.

As shown, modes information data 636 may be received via symbol-run VLCand/or proxy VLC encoder for modes module 632. In some examples, anentropy encoding technique may be determined for modes information data636 based on a parameter, parameters or characteristics of modesinformation data 636 or other system parameters (e.g., compressionefficiency, a characteristic of the data, and so on), as discussedherein. The determined entropy encoding technique may be applied tomodes information data 636 to generate compressed modes information data642. As shown, in some examples, the entropy encoding technique formodes information data 636 may be one of an adaptive symbol-run variablelength coding technique or an adaptive proxy variable length codingtechnique, as described above.

As shown, split/not-split information data 637 may be received viasymbol-run VLC and/or proxy VLC encoder for splits module 633. In someexamples, an entropy encoding technique may be determined forsplit/not-split information data 637 based on a parameter, parameters orcharacteristics of split/not-split information data 637 or other systemparameters (e.g., compression efficiency, a characteristic of the data,and so on), as discussed herein. The determined entropy encodingtechnique may be applied to split/not-split information data 637 togenerate compressed split/not-split information data 643. As shown, insome examples, the entropy encoding technique for split/not-splitinformation data 637 may be one of an adaptive symbol-run variablelength coding technique or an adaptive proxy variable length codingtechnique, as described above.

As shown, prediction reference information data 638 may be received viasymbol-run VLC and/or proxy VLC encoder for reference types module 634.In some examples, an entropy encoding technique may be determined forprediction reference information data 638 based on a parameter,parameters or characteristics of prediction reference information data638 or other system parameters (e.g., compression efficiency, acharacteristic of the data, and so on), as discussed herein. Thedetermined entropy encoding technique may be applied to predictionreference information data 638 to generate compressed predictionreference information data 644. As shown, in some examples, the entropyencoding technique for prediction reference information data 638 may beone of an adaptive symbol-run variable length coding technique or anadaptive proxy variable length coding technique, as described above.

As shown, encoder module 404 may include symbol-run VLC and/or proxy VLCencoder for coded/not-coded data 641 and may receive video data 414.Video data 414 may have a data type such that video data 411 maycoded/not-coded data. For example, a partition (or sub-partition) may becoded if it has any nonzero transform coefficients and a partition (orsub-partition) may be not-coded if it has all zero transformcoefficients. In some examples, coded/not-coded data may not be neededfor partitions having an intra or skip mode. In some examples,coded/not-coded data may be needed for partitions having an auto, inter,or multi mode. In some examples, an entropy encoding technique may bedetermined for video data 414 based on a parameter, parameters orcharacteristics of video data 414 or other system parameters. In someexamples, the entropy encoding technique for video data 414 may be oneof an adaptive symbol-run variable length coding technique or anadaptive proxy variable length coding technique, as described above. Thedetermined entropy encoding technique may be applied to video data 414to generate compressed video data 424. In some examples, encoder module404 may determine which technique provides the greatest compressionefficiency as discussed such that the parameter(s) associated with videodata 411 may be the number of bits needed for each coding technique orthe like. In some examples, the parameter associated with video data 411may be based on a characteristic of the video data such that thedetermined encoding technique may be adaptive to the received videodata.

As shown, in some examples, encoder module 405 may include an adaptivevector VLC encoder for transform coefficients module 651 and/or anadaptive 1D and 2D VLC encoder for transform coefficients module 652.

As shown, adaptive vector VLC encoder for transform coefficients module651 may receive transform coefficients data 653, which may includetransform coefficient data for blocks or partitions or sub-partitionshaving a block or partition or sub-partition size of 2 in one dimension(e.g., 2×K partitions or K×2 sized partitions). An adaptive vectorvariable length coding technique may be applied to transformcoefficients data 653 to generate compressed transform coefficients data655. In some examples, the adaptive vector variable length codingtechnique may include a quad-tree division of the block or partition orthe, representing each quadrant generated via the quad-tree divisionwith a single vector codeword that represents all coefficients with asingle index value, and entropy coding the codeword using a variablelength coding technique or the like.

Also as shown, adaptive 1D and 2D VLC encoder for transform coefficientsmodule 652 may receive transform coefficient data 654 and may implementan adaptive 1-dimensional variable length coding for blocks orpartitions or sub-partitions of size 4×4 and an adaptive 2-dimensionalvariable length coding for all other block or partition or sub-partitionsizes (e.g., 4×8, 8×4, 8×8, 16×16, 32×32, 64×64, and so on) to generatecompressed transform coefficient data 656. As discussed, transformcoefficient data 653, 654 may result from a forward transform ofrectangular (or square or the like) partitions of pixel data orrectangular (or square or the like) of pixel difference valuesimplemented via adaptive transform module 108 followed by a quantizationof the resulting coefficients via adaptive quantize module 109. In someexamples, the transform coefficient data may be scanned to convert it toa 1-dimensional frequency ordered partition via encoder module 405. Suchconversion may be highly adaptive any partition size (e.g., 24 or moreor 32 or more partition sizes), different data types (e.g., discretecosine transform coefficients or hybrid parametric Haar transformcoefficients or the like of either intra or inter partitions), and/ordifferent quantizer set characteristics (e.g., various combinations ofquantizer parameters and/or matrices). Further, a block or partition orsub-partition may belong to different picture types: I-picture (e.g.,intra compensation only), P-picture (e.g., predictive) or F-picture(e.g., functional) and/or may represent different types of signal ordata (e.g., luma or chroma or the like), which may be quantized withdifferent quantizer setting.

As shown, encoder module 406 may include a motion vector predictormodule 661, an adaptive VLC encoder for motion vector differences module662, and/or a differencer 663. As shown, encoder module 406 may receivevideo data 416, which may include motion vector data, via motion vectorpredictor module 661. Motion vector predictor module 661 may performmotion vector prediction based on video data 416 (e.g., the motionvector data of video data 516) using original motion vector(s) togenerate associated predicted motion vector(s). In some examples, themotion vector prediction may be based on immediate neighbors to left,right, above, or below the motion vector being predicted. In someexamples, other spatial neighbors that may share the same or similarcharacteristics may be used. For example, a number of different types ofprediction may be adaptively selected and the selection information mayprovided to decoder 200 via bitstream 111. Differencer 663 maydifference the predicted motion vector(s) and the original motionvector(s) to generate difference motion vector(s) for entropy coding. Asshown adaptive VLC encoder for motion vector differences module 662 mayapply an adaptive variable length coding technique to the differencemotion vector(s) to generate compressed video data 526. In someexamples, differential (e.g., difference) motion vectors may have twicethe range of original motion vectors. Further ⅛^(th) pel precisionmotion compensation may expand the range of the difference motion vectorby a factor of 8. In some examples, to address such expansion,classification of large space(s) into smaller subintervals and indexingof vectors inside the subinterval may be used.

As shown, encoder module 407 may include adaptive VLC and/or arithmeticencoder for intra-prediction and directions data module 671 and mayreceive video data 417. Video data 517 may have a data type such thatvideo data 417 may include intra-prediction type or intra-predictiondirection data. In some examples, an entropy encoding technique may bedetermined for video data 417 based on a parameter, parameters orcharacteristics of video data 417 or other system parameters (e.g.,compression efficiency or the like) as discussed herein. In someexamples, the entropy encoding technique for video data 417 may be oneof an adaptive variable length coding technique or an arithmetic codingtechnique, as described above. The determined entropy encoding techniquemay be applied to video data 417 to generate compressed video data 427.In some examples, the adaptive variable length coding technique mayinclude a content adaptive variable length coding based on the contentof video data 417. In some examples, the arithmetic coding technique mayinclude a content adaptive binary arithmetic coding based on the contentof video data 417. In some examples, video data 417 may support 9 ormore prediction directions and a variety of prediction types includingplanar, Binary Tree Predictive Coding (BTPC), or the like.

As shown in FIG. 6 and discussed above with respect to FIG. 4, theoutput encoder modules 401-407 (via the associated sub-modules) may beinput to bitstream assembler 408, which may output a multiplexedbitstream formatted per the bitstream syntax, as discussed above.

FIG. 7 is an illustrative diagram of an example entropy decoder module202, arranged in accordance with at least some implementations of thepresent disclosure. As shown and as discussed above with respect to FIG.5, entropy decoder module 202 may include headers, parameters and mapsdata decoder module 501, picture partitions, prediction partitions, andcoding partitions decoder module 502, coding modes and reference typesdecoder module 503, coded/not-coded data decoder module 504, transformcoefficients decoder module 505, motion vector and differential motionvector decoder module 506, intra-prediction and direction data decodermodule 507, and/or bitstream disassembler module 508. In some examples,entropy decoder module 202 (and the pertinent sub-modules) may performan inverse technique with respect to entropy encoder module 110 (and thepertinent sub-modules) such that there may be a one-to-onecorrespondence between encoder modules (and sub-modules) and decodermodules (and sub-modules)

As shown, bitstream disassembler module 508 may receive input bitstream201. In some examples, input bitstream 201 may be a valid bitstream suchas, for example, a valid next generation video (NGV) coded bitstream,which may follow a NGV bitstream syntax specification or any validbitstream as discussed herein. As discussed with respect to FIG. 5,bitstream disassembler module 508 may disassemble input bitstream 201 todetermine compressed video data 511-517, which may each have one or morecomponent parts as discussed further below. For example, bitstreamdisassembler module 508 may use a predefined syntax or specification todivide input bitstream 201 into component compressed video data 511-517by data type for decompression via decoder modules 501-507. In someexamples, bitstream disassembler module 508 may perform an inverseoperation with respect to bitstream assembler module 308. In someexamples, the disassembling of input bitstream 201 may be characterizedas a de-multiplexing.

As shown, decoder module 501 may include adaptive VLC, symbol-run VLC,and/or proxy VLC decoder for headers, parameters, and maps data module711 and may receive compressed video data 511. In some examples,compressed video data 511 may include header data (e.g., sequence and/orpicture level bitstream headers), morphing parameters, synthesizingparameters, and/or global maps data entropy encoded using one of anadaptive symbol-run variable length coding technique, an adaptive proxyvariable length coding technique, or a variable length coding table ortables compression technique. In some examples, adaptive VLC, symbol-runVLC, and/or proxy VLC decoder for headers, parameters, and maps datamodule 711 may determine an entropy decoding technique applicable tocompressed video data 511 and decode compressed video data 511 using theapplicable technique to generate video data 521. In some examples, theapplicable technique may be determined based on an indicator, parameter,header data, or the like conveyed via input bitstream 201.

As shown, decoder module 502 may include adaptive codebook VLC decoderfor picture partitions module 721, symbol-run VLC and/or proxy VLCdecoder for intra-prediction partitions data module 722, and/orsymbol-run VLC and/or proxy VLC decoder for inter-prediction partitionsand coding partitions data module 723 and may receive compressed videodata 512.

As shown, compressed picture slices or regions data 724 may be receivedvia adaptive codebook VLC decoder for picture partitions module 721. Insome examples, adaptive codebook VLC decoder for picture partitionsmodule 721 may apply adaptive codebook variable length coding tocompressed picture slices or regions data 724 to generate picture slicesor regions data 727. As discussed, adaptive codebook variable lengthcoding may include content adaptive variable length coding using ancodebook adaptive to the content of compressed picture slices or regionsdata 724 or other system parameters or the like. In some examples, thecodebook may be implemented via adaptive codebook VLC decoder forpicture partitions module 721.

As shown, symbol-run VLC and/or proxy VLC decoder for intra-predictionpartitions data 722 may receive compressed intra-prediction partitiondata 725. In some examples, compressed intra-prediction partition data725 may include compressed intra-prediction partition data entropyencoded using one of an adaptive symbol-run variable length codingtechnique or an adaptive proxy variable length coding technique, asdescribed herein. In some examples, symbol-run VLC and/or proxy VLCdecoder for intra-prediction partitions data 722 may determine anentropy decoding technique applicable to compressed intra-predictionpartition data 725 and entropy decode compressed intra-predictionpartition data 725 using the applicable technique to generateintra-prediction partition data 728. In some examples, the applicablecoding technique may be determined based on an indicator, parameter,header data, or the like conveyed via input bitstream 201.

As shown, symbol-run VLC and/or proxy VLC decoder for inter-predictionpartitions and coding partitions data 723 may receive compressedinter-prediction and coding partition data 726. In some examples,compressed inter-prediction and coding partition data 726 may includecompressed inter-prediction and coding partition data entropy encodedusing one of an adaptive symbol-run variable length coding technique oran adaptive proxy variable length coding technique, as described herein.In some examples, symbol-run VLC and/or proxy VLC decoder forinter-prediction partitions and coding partitions data 723 may determinean entropy decoding technique applicable to compressed inter-predictionand coding partition data 726 and entropy decode compressedinter-prediction and coding partition data 726 using the applicabletechnique to generate inter-prediction and coding partition data 729. Insome examples, the applicable coding technique may be determined basedon an indicator, parameter, header data, or the like conveyed via inputbitstream 201.

As shown, decoder module 503 may include adaptive VLC decoder for jointmodes and splits module 731, symbol-run VLC and/or proxy VLC decoder formodes module 732, symbol-run VLC and/or proxy VLC decoder for splitsmodule 733, and/or symbol-run VLC and/or proxy VLC decoder for referencetypes module 734 and may receive compressed video data 513.

As discussed above with respect to encoder module 403, in some examples,modes and splits data may be coded jointly and, in some examples, modesand splits data may be coded separately. In some examples, adaptive VLCdecoder for joint modes and splits module 731 may decode jointly codeddata, and symbol-run VLC and/or proxy VLC decoder for modes module 732and symbol-run VLC and/or proxy VLC decoder for splits module 733 maydecode separately coded data. In some examples, whether data is jointlyor separately coded may be indicated via input bitstream 201.

As shown, compressed joint coding of modes and splits data 735 may bereceived via adaptive VLC decoder for joint modes and splits module 731.In some examples, adaptive VLC decoder for joint modes and splits module731 may apply adaptive variable length coding to compressed joint codingof modes and splits data 735 to generate joint coding of modes andsplits data 739. As discussed, adaptive variable length coding may becontent adaptive variable length coding adaptive to the content ofcompressed joint coding of modes and splits data 735 or other systemparameters or the like.

As shown, symbol-run VLC and/or proxy VLC decoder for modes module 732may receive compressed modes information data 736. In some examples,compressed modes information data 736 may include compressed modesinformation data entropy encoded using one of an adaptive symbol-runvariable length coding technique or an adaptive proxy variable lengthcoding technique, as described herein. In some examples, symbol-run VLCand/or proxy VLC decoder for modes module 732 may determine an entropydecoding technique applicable to compressed modes information data 736and entropy decode compressed modes information data 736 using theapplicable technique to generate modes information data 742. In someexamples, the applicable coding technique may be determined based on anindicator, parameter, header data, or the like conveyed via inputbitstream 201.

As shown, symbol-run VLC and/or proxy VLC decoder for splits module 733may receive compressed splits information data 737. In some examples,compressed splits information data 737 may include compressed splitsinformation data entropy encoded using one of an adaptive symbol-runvariable length coding technique or an adaptive proxy variable lengthcoding technique, as described herein. In some examples, symbol-run VLCand/or proxy VLC decoder for splits module 733 may determine an entropydecoding technique applicable to compressed splits information data 737and entropy decode compressed splits information data 737 using theapplicable technique to generate splits information data 743. In someexamples, the applicable coding technique may be determined based on anindicator, parameter, header data, or the like conveyed via inputbitstream 201.

As shown, symbol-run VLC and/or proxy VLC decoder for reference typesmodule 734 may receive compressed reference types information data 738.In some examples, compressed reference types information data 738 mayinclude compressed reference types information data entropy encodedusing one of an adaptive symbol-run variable length coding technique oran adaptive proxy variable length coding technique, as described herein.In some examples, symbol-run VLC and/or proxy VLC decoder for referencetypes module 734 may determine an entropy decoding technique applicableto compressed reference types information data 738 and entropy decodecompressed reference types information data 738 using the applicabletechnique to generate reference types information data 744. In someexamples, the applicable coding technique may be determined based on anindicator, parameter, header data, or the like conveyed via inputbitstream 201.

As shown, decoder module 504 may include symbol-run VLC and/or proxy VLCdecoder for coded/not-coded data module 741 and may receive compressedvideo data 514. In some examples, compressed video data 514 may includecoded/not-coded, as discussed herein, entropy encoded using one of anadaptive symbol-run variable length coding technique or an adaptiveproxy variable length coding technique. In some examples, symbol-run VLCand/or proxy VLC decoder for coded/not-coded data module 741 maydetermine an entropy decoding technique applicable to compressed videodata 514 and decode compressed video data 514 using the applicabletechnique to generate video data 524. In some examples, the applicabletechnique may be determined based on an indicator, parameter, headerdata, or the like conveyed via input bitstream 201. In some examples,the applicable technique may be one of an adaptive symbol-run variablelength coding technique or an adaptive proxy variable length codingtechnique.

As shown, decoder module 505 may include adaptive vector VLC decoder fortransform coefficients module 751 and/or adaptive 1D and 2D VLC decoderfor transform coefficients module 752 and may receive compressed videodata 515.

As shown, adaptive vector VLC decoder for transform coefficients module751 may receive compressed transform coefficients data 753, which mayinclude compressed transform coefficients data for blocks or partitionsor sub-partitions having a size of 2 in one dimension (e.g., 2×Kpartitions or K×2 sized partitions), as discussed herein. In someexamples, adaptive vector VLC decoder for transform coefficients module751 may apply an adaptive vector variable length coding technique toentropy decode compressed transform coefficients data 753 to generatetransform coefficients data 755. As discussed, In some examples, theadaptive vector variable length coding technique may include using avariable length codeword to generate all coefficients of a quad-treedivision of the block, which may be generated via merging the quad-treedivision.

As shown, adaptive 1D and 2D VLC decoder for transform coefficientsmodule 752 may receive compressed transform coefficients data 754, whichmay include compressed transform coefficients data for blocks orpartitions or sub-partitions of size 4×4 and all other block orpartition or sub-partition sizes (e.g., 4×8, 8×4, 8×8, 16×16, 32×32,64×64, and so on). In some examples, adaptive 1D and 2D VLC decoder fortransform coefficients module 752 may apply an adaptive 1-dimensionalvariable length coding for blocks or partitions or sub-partitions ofsize 4×4 and an adaptive 2-dimensional variable length coding for allother block or partition or sub-partition sizes (e.g., 4×8, 8×4, 8×8,16×16, 32×32, 64×64, and so on) to entropy decode compressed transformcoefficients data 754 to generate transform coefficients data 756.

As shown, decoder module 506 may include an adaptive VLC decoder formotion vector differences module 762, a motion vector predictor 761 andan adder 763, and may receive compressed video data 516. In someexamples, adaptive VLC decoder for motion vector differences module 762may decode compressed video data 516 to generate motion vectordifferences. Furthermore, motion vector predictor 761 may generateprediction motion vectors using previously decoded neighboring motionvectors in analogy to the techniques discussed with respect to motionvector predictor module 661. As shown, decoded difference motionvector(s) may be added via adder 763 to prediction motion vector(s) togenerate reconstructed motion vector(s), which may be output as a partof video data 526 and further used to perform motion vector predictionfor other motion vectors via motion vector predictor module 761.

As shown, decoder module 507 may include adaptive VLC and/or arithmeticdecoder for intra-prediction type and direction module 771 and mayreceive compressed video data 517. In some examples, compressed videodata 517 may include intra-prediction type and direction data entropyencoded using one of an adaptive VLC technique or an arithmetic codingtechnique. In some examples, adaptive VLC and/or arithmetic decoder forintra-prediction type and direction module 771 may determine an entropydecoding technique applicable to compressed video data 517 and decodecompressed video data 517 using the applicable technique to generatevideo data 527. In some examples, the applicable technique may bedetermined based on an indicator, parameter, header data, or the likeconveyed via input bitstream 201.

As discussed, video data 521-527 (including various sub-components asdiscussed) may be transmitted to various components of decoder 200.Further, as discussed, decoder 200 may use entropy decoded video data521-527 to generate video frames, which may be output, via display video219, for presentment or display via a display device for a user.

FIG. 8 is a flow diagram illustrating an example process 800, arrangedin accordance with at least some implementations of the presentdisclosure. Process 800 may include one or more operations, functions oractions as illustrated by one or more of operations 802,804,806, and/or808. Process 800 may form at least part of a next generation videocoding process. By way of non-limiting example, process 800 may form atleast part of a next generation video encoding process as undertaken byencoder system 100 of FIG. 1 and/or entropy encoder module 110 of FIG. 5or 7.

Process 800 may begin at operation 802, “Obtain First and Second VideoData of Different Types for Entropy Encoding”, where first and second(or additional) video data of different types may be obtained orreceived for entropy coding. For example, two or more of video data411-417 (and/or any sub-components of video data 511-517) may bereceived. In some examples, video data 411-417 may be received viaadaptive entropy encoder 110. As discussed, the first and second (oradditional) video data may be of different types such as any of thetypes or sub-component types as discussed with respect to video data411-417 or elsewhere herein.

Process 800 may continue at operation 804, “Determine an EntropyEncoding Technique for the First Video Data”, where a first entropyencoding technique may be determined for the first video data. Asdiscussed, in some examples, one or more of encoder modules 401-407 maydetermine a coding technique for video data 411-417 from various codingtechnique options. For example, encoder module 401 may determine acoding technique for video data 411 from one of an adaptive symbol-runvariable length coding technique, an adaptive proxy variable lengthcoding technique, or a variable length coding table or tablescompression technique. Further, encoder module 402 may determine acoding technique for intra-prediction partition data 625 of video data402 from one of an adaptive symbol-run variable length coding techniqueand an adaptive proxy variable length coding technique, and so on. Awide range of examples have been provided herein with respect to FIGS. 4and 6 and will not be repeated here for the sake of brevity. Asdiscussed, in some examples, two ore more types of video data may bereceived. In some examples, an entropy encoding technique may bedetermined for two, three, or more types of video data as describedherein.

As discussed, an entropy encoding technique may be determined for videodata. In some examples, a selected coding table associated with thefirst encoding technique may be determined from two or more availabletables. For example, the table selection may be made based on a bitcount comparison, a coding efficiency, or the like between the availabletables. For example, any of the described entropy encoding techniquesdiscussed herein may have multiple available tables. In some examples, atable indicator may be generated and provided in bitstream 900indicating the selected table for the video data. For example, bitstream900 may include indicators indicating the selected encoding techniqueand the selected coding table. Further an indicator or indicatorsassociated with the length of the video data may be included intobitstream 900. For example, the length or number of tuples in the videodata (e.g. the length of the string of video) may be provided and anyremainder portion (e.g., if the length is not evenly divided intotuples) may be coded using a bit map technique and provided into theoutput bitstream.

Process 800 may continue at operation 806, “Entropy Encode the FirstVideo Data using the First Entropy Encoding Technique and Entropy Encodethe Second Video Data to Generate First and Second Compressed VideoData”, where the first video data may be entropy encoded using the firstentropy encoding technique to generate first compressed video data andthe second video data may be compressed using a second entropy encodingtechnique to generate first and second compressed video data.

Process 800 may continue at operation 808, “Assemble the First andSecond Compressed Video Data to Generate an Output Bitstream”, where thefirst compressed video data and the second compressed video data may beassembled to generate an output bitstream. For example, bitstreamassembler 508 may assemble or multiplex the first compressed video dataand the second compressed video data to generate output bitstream 111 asdiscussed herein.

As discussed, in some examples, output bitstream 111 may be multiplexedwith an audio stream (coded or uncoded) to generate a multiplexedaudio-visual stream. Further, as discussed, in some examples, one ormore of the video data may be preprocessed or otherwise manipulatedprior to entropy coding. For example, for motion vector data, motionvector prediction may be performed to generate predicted motionvector(s), the predicted motion vector(s) may be differenced with theoriginal motion vector(s) to generate difference motion vector(s), andthe difference motion vector(s) may be entropy coded as discussedherein. Further, in some examples, each of the seven encoder modules401-407 may be implemented simultaneously to operate on seven types ofvideo data 411-417. Process 800 may be implemented via adaptive entropyencoder module 110 as discussed herein. Further, process 800 may berepeated either in serial or in parallel on any number of instantiationsof video data.

As discussed, video data of different types may be entropy coded using avariety of determined (or predetermined) adaptive entropy codingtechniques to generate compressed video data. The compressed video datamay be assembled to generate an output bitstream.

FIG. 9 illustrates an example bitstream 900, arranged in accordance withat least some implementations of the present disclosure. In someexamples, bitstream 900 may correspond to output bitstream 111 as shownin FIGS. 1, 3 a, 4, and 6 and/or input bitstream 201 as shown in FIGS.2, 3 b, 5, and 7. Although not shown in FIG. 9 for the sake of clarityof presentation, in some examples bitstream 900 may include a headerportion and a data portion. In various examples, bitstream 900 mayinclude data, indicators, index values, mode selection data, or the likeassociated with encoding compressed video as discussed herein. As shown,in some examples, bitstream 900 may include indicator data 901,compressed video data 421, compressed video data 422, compressed videodata 423, compressed video data 424, compressed video data 425,compressed video data 426, and/or compressed video data 427. Theillustrated data may be in any order in bitstream 900 and may beadjacent or separated by any other of a variety of additional data forcoding video. As discussed, bitstream 900 may also include indicatorsindicating the selected encoding technique and the selected coding table(e.g., in indicator data 901). Further an indicator or indicatorsassociated with the length of the video data may be included intobitstream 900. For example, the length or number of tuples in the videodata (e.g. the length of the string of video) may be provided and anyremainder portion (e.g., if the length is not evenly divided intotuples) may be coded using a bit map technique and provided into theoutput bitstream.

In some examples, compressed video data 421-427 may include anycompressed video data encoded via any technique as discussed herein. Insome examples, indicator data 901 may include header data, modeindicator data, and/or data indicating entropy encoding techniquesassociated with compressed video data 421-427. For example, indicatordata 901 may indicate an entropy coding technique used for compressingvideo data 421, an entropy coding technique used for compressingportions of video data 422, an entropy coding technique used forcompressing portions of video data 423, an entropy coding technique usedfor compressing video data 424, and/or an entropy coding technique usedfor compressing video data 427, as discussed herein with respect toFIGS. 4 and 6.

As discussed, bitstream 900 may be generated by an encoder such as, forexample, encoder 100 and/or received by a decoder 200 for decoding suchthat video frames may be presented via a display device.

FIG. 10 is a flow diagram illustrating an example process 1000, arrangedin accordance with at least some implementations of the presentdisclosure. Process 1000 may include one or more operations, functionsor actions as illustrated by one or more of operations 1002,1004, 1006,and/or 1008. Process 1000 may form at least part of a next generationvideo coding process. By way of non-limiting example, process 1000 mayform at least part of a next generation video decoding process asundertaken by decoder system 200 of FIG. 2.

Process 1000 may begin at operation 1002, “Receive Entropy EncodedBitstream”, where an entropy encoded bitstream may be received. Forexample, a bitstream encoded as discussed herein may be received at avideo decoder. In some examples, bitstream 100 or 900 or the like may bereceived via decoder 200.

Process 1000 may continue at operation 1004, “Disassemble the EntropyEncoded Bitstream to First Compressed Video Data and Second CompressedVideo Data”, where the received bitstream may be disassembled todetermine different types of compressed video data. For example,bitstream 201 may be disassembled via bitstream disassembler 508 togenerate compressed video data 511-517 (and any sub-component videodata) as illustrated in FIG. 7. For example, the disassembled compressedvideo data may include first compressed video and second compressedvideo data.

Process 1000 may continue at operation 1006, “Entropy Decode the Firstand Second Compressed Video Data to Generate First Video Data and SecondVideo Data”, where first and second compressed video data may be entropydecoded to generate (decompressed) first and second video data. Asdiscussed, in some examples, one or more of decoder modules 501-507 maydetermine a coding technique for compressed video data 511-517 fromvarious coding technique options. For example, decoder module 501 maydetermine a coding technique for compressed video data 511 from one ofan adaptive symbol-run variable length coding technique, an adaptiveproxy variable length coding technique, or a variable length codingtable or tables compression technique based on an indicator orindicators provided via the input bitstream. Further, decoder module 502may determine a coding technique for compressed intra-predictionpartition data 725 of video data 512 from one of an adaptive symbol-runvariable length coding technique and an adaptive proxy variable lengthcoding technique, and so on. A wide range of examples have been providedherein with respect to FIGS. 5 and 7 and will not be repeated here forthe sake of brevity. As discussed, in some examples, two ore more typesof video data may be received. In some examples, an entropy decodingtechnique may be determined for two, three, or more types of video dataas described herein.

Process 1000 may continue at operation 1008, “Decode the First andSecond Video Data to Generate a Video Frame”, where the first and secondvideo data (and any other entropy decoded video data) may be decoded togenerate video frame(s). The video frames may be suitable forpresentment to a user via a display device, for example.

As discussed, in some examples, one or more of the entropy decoded videodata may be post-processed or otherwise manipulated prior to furtherdecoding. For example, entropy decoded difference motion vector(s) maybe added to prediction motion vector(s) to generate reconstructed motionvector(s), which may be output for use in motion vector prediction(e.g., inter-prediction) via decoder 200.

Various components of the systems described herein may be implemented insoftware, firmware, and/or hardware and/or any combination thereof. Forexample, various components of encoder 100 or encoder 200 may beprovided, at least in part, by hardware of a computing System-on-a-Chip(SoC) such as may be found in a computing system such as, for example, asmart phone. Those skilled in the art may recognize that systemsdescribed herein may include additional components that have not beendepicted in the corresponding figures. For example, the systemsdiscussed herein may include additional components such as bit streammultiplexer or de-multiplexer modules and the like that have not beendepicted in the interest of clarity.

Some additional and/or alternative details related to process 800, 1000and other processes discussed herein may be illustrated in one or moreexamples of implementations discussed herein and, in particular, withrespect to FIG. 11 below.

FIG. 11 is an illustrative diagram of an example video coding system1610 and video coding process 1100 in operation, arranged in accordancewith at least some implementations of the present disclosure. In theillustrated implementation, process 1100 may include one or moreoperations, functions or actions as illustrated by one or more ofactions 1100-1112. By way of non-limiting example, process 1100 will bedescribed herein with reference to example video coding system 1610including encoder 100 of FIG. 1 and decoder 200 of FIG. 2, as isdiscussed further herein below with respect to FIG. 16. In variousexamples, process 1100 may be undertaken by a system including both anencoder and decoder or by separate systems with one system employing anencoder (and optionally a decoder) and another system employing adecoder (and optionally an encoder). It is also noted, as discussedabove, that an encoder may include a local decode loop employing a localdecoder as a part of the encoder system.

In the illustrated implementation, video coding system 1610 may includelogic circuitry 1150, the like, and/or combinations thereof. Forexample, logic circuitry 1150, may include encoder 100 and may includeany modules as discussed with respect to FIG. 1 and/or FIGS. 3 and 5 anddecoder 200 and may include any modules as discussed with respect toFIG. 2 and/or FIGS. 4 and 6. Although video coding system 1610, as shownin FIG. 16, may include one particular set of blocks or actionsassociated with particular modules, these blocks or actions may beassociated with different modules than the particular module illustratedhere. Although process 1100, as illustrated, is directed to encoding anddecoding, the concepts and/or operations described may be applied toencoding and/or decoding separately, and, more generally, to videocoding.

Process 1100 may begin at operation 1101, “Obtain Video Data ofDifferent Types”, where video data of different types may be receivedfor entropy encoding. For example two or more types of video data may bereceived for entropy encoding. For example, two or more of video data411-417 (and/or any sub-components of video data 411-417) may bereceived via adaptive entropy encoder 110. As discussed, the first andsecond (or additional) video data may be of different types such as anyof the types or sub-component types as discussed with respect to videodata 411-417 or elsewhere herein.

Process 1100 may continue from operation 1101 to operation 1102,“Determine Entropy Encoding Technique(s) for one or more Types of VideoData”, where entropy encoding technique(s) may be determined for one ormore of the video data types. As discussed, in some examples, one ormore of encoder modules 401-407 may determine a coding technique forvideo data 411-417 from various coding technique options. For example,encoder module 401 may determine a coding technique for video data 411from one of an adaptive symbol-run variable length coding technique, anadaptive proxy variable length coding technique, or a variable lengthcoding table or tables compression technique. Further, encoder module402 may determine a coding technique for intra-prediction partition data625 of video data 402 from one of an adaptive symbol-run variable lengthcoding technique and an adaptive proxy variable length coding technique,and so on as discussed herein.

Process 1100 may continue at operation 1103, “Entropy Encode the VideoData using the Determined Technique(s)”, the video data may be entropyencoded using the determined technique(s). For example, first video datamay be entropy encoded using a first entropy encoding technique togenerate first compressed video data. In some examples, second, third,or more additional video data may be entropy encoded using second, thirdor more additional entropy encoding techniques as discussed herein togenerate first, second, third, and so on respective compressed videodata.

Process 1100 may continue from operation 1103 to operation 1104,“Assemble the Compressed Video Data”, where the compressed video data ofdifferent types may be assembled to generate an output bitstream. Forexample, bitstream assembler 408 may assemble or multiplex thecompressed video data to generate output bitstream 111 as discussedherein.

Process 1100 may continue from operation 1104 to operation 1105,“Optionally Multiplex Video Bitstream with Audio Stream”, where thevideo bitstream may be optionally multiplexed with a coded or uncodedaudio stream to generate an audio-visual bitstream.

Process 1100 may continue from operation 1105 or operation 1104 tooperation 1106, “Transmit Bitstream”, where the bitstream may betransmitted. For example, video coding system 1610 may transmit outputbitstream 111 or bitstream 800 or the like via an antenna 1102 (pleaserefer to FIG. 16).

Operations 1101-1106 may provide for video encoding and bitstreamtransmission techniques, which may be employed by an encoder system asdiscussed herein. The following operations, operations 1107-1112 mayprovide for video decoding and video display techniques, which may beemployed by a decoder system as discussed herein.

Process 1100 may continue at operation 1107, “Receive Bitstream”, wherethe encoded bitstream may be received. For example, input bitstream100,201, or bitstream 800 or the like may be received via decoder 200.In some examples, the bitstream may include different types of entropyencoded data as discussed herein.

Process 1100 may continue from operation 1107 to operation 1108,“Disassamble Bitstream to Generate Compressed Video Data of DifferentTypes”, where the received bitstream may be disassembled to determinedifferent types of compressed video data. For example, bitstream 201 maybe disassembled via bitstream disassembler 508 to generate compressedvideo data 511-517 (and any sub-component video data) as illustrated inFIG. 7. For example, the disassembled compressed video data may includefirst, second, third, or more compressed video data.

Process 1100 may continue from operation 1108 to operation 1109,“Determine Entropy Decoding Technique(s) for one or more Types ofCompressed Video Data”, entropy decoding technique(s) may be determinedfor one or more of the compressed video data types. In some examples,the entropy decoding technique(s) may be determined based on a flag orindicator or the like conveyed via the received bitstream. As discussed,in some examples, one or more of decoder modules 501-507 may determine acoding technique for compressed video data 511-517 from various codingtechnique options. For example, decoder module 401 may determine acoding technique for compressed video data 511 from one of an adaptivesymbol-run variable length coding technique, an adaptive proxy variablelength coding technique, or a variable length coding table or tablescompression technique based on an indicator or indicators provided viathe input bitstream. Further, decoder module 502 may determine a codingtechnique for compressed intra-prediction partition data 725 of videodata 512 from one of an adaptive symbol-run variable length codingtechnique and an adaptive proxy variable length coding technique, and soon. A wide range of examples have been provided herein and will not berepeated here for the sake of brevity. As discussed, in some examples,two ore more types of video data may be received. In some examples, anentropy decoding technique may be determined for two, three, or moretypes of video data as described herein.

Process 1100 may continue from operation 1109 to operation 1610,“Entropy Decode the Compressed Video Data”, where the compressed videodata may be entropy decoded based on the determined entropy decodingtechniques. For example, compressed video data 511-517 may be entropydecoded via decode modules 501-507.

Process 1100 may continue from operation 1610 to operation 1111, “Decodethe Entropy Decoded Video Data to Generate Video Frame(s)”, where thefirst and second video data (and any other entropy decoded video data)may be decoded to generate video frame(s). The video frames may besuitable for presentment to a user via a display device, for example.For example, the video frame may be determined based on theimplementation of decode techniques discussed with respect to decoder200.

Process 1100 may continue from operation 1111 to operation 1112,“Transmit Video Frames for Presentment via a Display Device”, wheregenerated video frame(s) may be transmitted for presentment via adisplay device. For example, the video frame(s) may be transmitted to adisplay device 1105 (as shown in FIG. 16) for presentment. In someexamples, display device 1105 may display the video frames to a user,for example.

While implementation of the example processes herein may include theundertaking of all operations shown in the order illustrated, thepresent disclosure is not limited in this regard and, in variousexamples, implementation of the example processes herein may include theundertaking of only a subset of the operations shown and/or in adifferent order than illustrated.

In addition, any one or more of the operations discussed herein may beundertaken in response to instructions provided by one or more computerprogram products. Such program products may include signal bearing mediaproviding instructions that, when executed by, for example, a processor,may provide the functionality described herein. The computer programproducts may be provided in any form of one or more machine-readablemedia. Thus, for example, a processor including one or more processorcore(s) may undertake one or more of the operations of the exampleprocesses herein in response to program code and/or instructions orinstruction sets conveyed to the processor by one or moremachine-readable media. In general, a machine-readable medium may conveysoftware in the form of program code and/or instructions or instructionsets that may cause any of the devices and/or systems described hereinto implement at least portions of the video systems as discussed herein.

As used in any implementation described herein, the term “module” refersto any combination of software logic, firmware logic and/or hardwarelogic configured to provide the functionality described herein. Thesoftware may be embodied as a software package, code and/or instructionset or instructions, and “hardware”, as used in any implementationdescribed herein, may include, for example, singly or in anycombination, hardwired circuitry, programmable circuitry, state machinecircuitry, and/or firmware that stores instructions executed byprogrammable circuitry. The modules may, collectively or individually,be embodied as circuitry that forms part of a larger system, forexample, an integrated circuit (IC), system on-chip (SoC), and so forth.For example, a module may be embodied in logic circuitry for theimplementation via software, firmware, or hardware of the coding systemsdiscussed herein.

FIG. 12 is a flow diagram illustrating an example process 1200, arrangedin accordance with at least some implementations of the presentdisclosure. Process 1200 may include one or more operations, functionsor actions as illustrated by one or more operations. Process 1200 mayform at least part of a next generation video coding process. By way ofnon-limiting example, process 1200 may form at least part of a nextgeneration video encoding process as undertaken by encoder system 100 ofFIG. 1 and/or any other encoder system or subsystems described herein.

Process 1200 may begin at operation 1202, “Receive Input Video Frames ofa Video Sequence”, where input video frames of a video sequence may bereceived via encoder 100 for example.

Process 1200 may continue at operation 1204, “Associate a Picture Typewith each Video Frame”, where a picture type may be associated with eachvideo frame in a group of pictures via content pre-analyzer module 102for example. For example, the picture type may be F/B-picture ,P-picture, or I-picture, or the like. In some examples, a video sequencemay include groups of pictures and the processing described herein(e.g., operations 1203 through 1211) may be performed on a frame orpicture of a group of pictures and the processing may be repeated forall frames or pictures of a group and then repeated for all groups ofpictures in a video sequence.

Process 1200 may continue at operation 1206, “Divide a Picture intoTiles and/or Super-fragments and Potential Prediction Partitionings”,where a picture may be divided into tiles or super-fragments andpotential prediction partitions via prediction partitions generator 105for example.

Process 1200 may continue at operation 1210, “For Potential PredictionPartitioning, Determine Potential Prediction Error”, where, for eachpotential prediction partitioning, a potential prediction error may bedetermined. For example, for each prediction partitioning (andassociated prediction partitions, prediction(s), and predictionparameters), a prediction error may be determined. For example,determining the potential prediction error may include differencingoriginal pixels (e.g., original pixel data of a prediction partition)with prediction pixels. In some examples, the associated predictionparameters may be stored. As discussed, in some examples, the predictionerror data partition may include prediction error data generated basedat least in part on a previously decoded frame generated using at leastone of a morphing technique or a synthesizing technique.

Process 1200 may continue at operation 1212, “Select PredictionPartitioning and Prediction Type and Save Parameters”, where aprediction partitioning and prediction type may be selected and theassociated parameters may be saved. In some examples, the potentialprediction partitioning with a minimum prediction error may be selected.In some examples, the potential prediction partitioning may be selectedbased on a rate distortion optimization (RDO).

Process 1200 may continue at operation 1214, “Perform Transforms onPotential Coding Partitionings”, where fixed or content adaptivetransforms with various block sizes may be performed on variouspotential coding partitionings of partition prediction error data. Forexample, partition prediction error data may be partitioned to generatea plurality of coding partitions. For example, the partition predictionerror data may be partitioned by a bi-tree coding partitioner module ora k-d tree coding partitioner module of coding partitions generatormodule 107 as discussed herein. In some examples, partition predictionerror data associated with an F/B- or P-picture may be partitioned by abi-tree coding partitioner module. In some examples, video dataassociated with an I-picture (e.g., tiles or super-fragments in someexamples) may be partitioned by a k-d tree coding partitioner module. Insome examples, a coding partitioner module may be chosen or selected viaa switch or switches. For example, the partitions may be generated bycoding partitions generator module 107.

Process 1200 may continue at operation 1216, “Determine the Best CodingPartitioning, Transform Block Sizes, and Actual Transform”, where thebest coding partitioning, transform block sizes, and actual transformsmay be determined. For example, various coding partitionings (e.g.,having various coding partitions) may be evaluated based on RDO oranother basis to determine a selected coding partitioning (which mayalso include further division of coding partitions into transform blockswhen coding partitions to not match a transform block size asdiscussed). For example, the actual transform (or selected transform)may include any content adaptive transform or fixed transform performedon coding partition or block sizes as described herein.

Process 1200 may continue at operation 1218, “Quantize and ScanTransform Coefficients”, where transform coefficients associated withcoding partitions (and/or transform blocks) may be quantized and scannedin preparation for entropy coding.

Process 1200 may continue at operation 1222, “Entropy Encode Dataassociated with Each Tile or Super-fragment Decode, Such As CodingPartition Indicator(s), Block Size Data, Transform Type Data, Quantizer(Qp), and Quantized Transform Coefficients, Motion Vectors and ReferenceType Data, Characteristic Parameters (e.g., mop, syp)”, where data maybe entropy encoded. For example, the entropy encoded data may includethe coding partition indicators, block size data, transform type data,quantizer (Qp), quantized transform coefficients, motion vectors andreference type data, characteristic parameters (e.g., mop, syp), thelike, and/or combinations thereof. Additionally or alternatively, theentropy encoded data may include prediction partitioning, predictionparameters, the selected coding partitioning, the selectedcharacteristics data, motion vector data, quantized transformcoefficients, filter parameters, selection data (such as mode selectiondata), and indictors.

Process 1200 may continue at operation 1223 “Apply DD/DB Filter,Reconstruct Pixel Data, Assemble into a Picture”, where deblockfiltering (e.g., DD or DB filters) may be applied, pixel data may bereconstructed, and assembled into a picture. For example, after a localdecode loop (e.g., including inverse scan, inverse transform, andassembling coding partitions), prediction error data partitions may begenerated. The prediction error data partitions may be added with aprediction partition to generate reconstructed prediction partitions,which may be assembled into tiles or super-fragments. The assembledtiles or super-fragments may be optionally processed via deblockfiltering and/or quality restoration filtering and assembled to generatea picture.

Process 1200 may continue at operation 1224 “Apply QR/LF Filter Save inReference Picture Buffers”, where quality restoration filtering (e.g.,QR or LF filtering) may be applied, and the assembled picture may besaved in reference picture buffers. For example, in addition to or inthe alternative to the DD/DB filtering, the assembled tiles orsuper-fragments may be optionally processed via quality restorationfiltering and assembled to generate a picture. The picture may be savedin decoded picture buffer 119 as a reference picture for prediction ofother (e.g., following) pictures.

Process 1200 may continue at operation 1225, “Apply AP/AM Filter,Determine Modifying (e.g., Morphing or Synthesizing) CharacteristicParameters for Generating Morphed or Synthesized Prediction Reference(s)and Perform Prediction(s)”, where, modifying (e.g., morphing orsynthesizing) characteristic parameters and prediction(s) may beperformed and adaptive motion filtering or adaptive precision filtering(e.g., AP/AM Filter) may be applied. For example, modifying (e.g.,morphing or synthesizing) characteristic parameters for generatingmorphed or synthesized prediction reference(s) may be generated andprediction(s) may be performed. Additionally, adaptive motion filteringor adaptive precision filtering may be applied at this point in theprocess.

As discussed, in some examples, inter-prediction may be performed. Insome examples, up to 4 decoded past and/or future pictures and severalmorphing/synthesis predictions may be used to generate a large number ofreference types (e.g., reference pictures). For instance in ‘inter’mode, up to nine reference types may be supported in P-pictures, and upto ten reference types may be supported for F/B-pictures. Further,‘multi’ mode may provide a type of inter prediction mode in whichinstead of 1 reference picture, 2 reference pictures may be used and P-and F/B-pictures respectively may allow 3, and up to 8 reference types.For example, prediction may be based on a previously decoded framegenerated using at least one of a morphing technique or a synthesizingtechnique. In such examples, and the bitstream (discussed below withrespect to operation 1212) may include a frame reference, morphingparameters, or synthesizing parameters associated with the predictionpartition.

Process 1200 may continue at operation 1229 “Optionally Apply EP Filterand/or Optionally apply FI/FP Filter”, where enhanced predictedpartition (e.g., EP Filtering) or FI/FP Filtering (e.g., fusionfiltering or fusion improvement filtering) may be optionally applied. Insome examples, a decision may be made regarding whether to utilize someform or FI/FP Filter (fusion improvement filtering/fusion filtering) ornot to use FI/FP Filtering. When some form or FI/FP Filter (e.g., fusionfiltering or fusion improvement filtering) is to be applied to theselected predicted partition the selected predicted partition and asecond selected predicted partition may be assembled to generate atleast a portion of an assembled picture. FI/FP Filtering may be appliedto filter the portion of the assembled picture. FI/FP Filteringparameters (e.g., filtering parameters or fusion improvement filteringparameters) associated with the FI/FP Filtering may be generated andsent to the entropy coder subsystem.

In implementations where both EP Filtering or FI/FP Filtering areavailable, an indicator may be generated that indicates to the decodersystem whether to use the enhanced predicted partition (e.g., EPFiltering) or the predicted partition data as the selected predictedpartition for the prediction partition.

Operations 1202 through 1229 may provide for video encoding andbitstream transmission techniques, which may be employed by an encodersystem as discussed herein.

FIG. 13 illustrates an example bitstream 1300, arranged in accordancewith at least some implementations of the present disclosure. In someexamples, bitstream 1300 may correspond to output bitstream 111 as shownin FIG. 1 and/or input bitstream 201 as shown in FIG. 2. Although notshown in FIG. 29 for the sake of clarity of presentation, in someexamples bitstream 1300 may include a header portion and a data portion.In various examples, bitstream 1300 may include data, indicators, indexvalues, mode selection data, or the like associated with encoding avideo frame as discussed herein.

As discussed, bitstream 1300 may be generated by an encoder such as, forexample, encoder 100 and/or received by a decoder 200 for decoding suchthat decoded video frames may be presented via a display device.

FIG. 14 is a flow diagram illustrating an example process 1400, arrangedin accordance with at least some implementations of the presentdisclosure. Process 1400 may include one or more operations, functionsor actions as illustrated by one or more operations. Process 1400 mayform at least part of a next generation video coding process. By way ofnon-limiting example, process 1400 may form at least part of a nextgeneration video decoding process as undertaken by decoder system 200and/or any other decoder system or subsystems described herein.

Process 1400 may begin at operation 1402, “Receive Encoded Bitstream”,where a bitstream may be received. For example, a bitstream encoded asdiscussed herein may be received at a video decoder. In some examples,bitstream 900 or 1300 may be received via decoder 200.

Process 1400 may continue at operation 1404, “Decode the Entropy EncodedBitstream to Determine Coding Partition Indicator(s), Block Size Data,Transform Type Data, Quantizer (Qp), Quantized Transform Coefficients,Motion Vectors and Reference Type Data, Characteristic Parameters (e.g.,mop, syp)”, where the bitstream may be decoded to determine codingpartition indicators, block size data, transform type data, quantizer(Qp), quantized transform coefficients, motion vectors and referencetype data, characteristic parameters (e.g., mop, syp), the like, and/orcombinations thereof. Additionally or alternatively, the entropy encodeddata may include prediction partitioning, prediction parameters, theselected coding partitioning, the selected characteristics data, motionvector data, quantized transform coefficients, filter parameters,selection data (such as mode selection data), and indictors.

Process 1400 may continue at operation 1406, “Apply Quantizer (Qp) onQuantized Coefficients to Generate Inverse Quantized TransformCoefficients”, where quantizer (Qp) may be applied to quantizedtransform coefficients to generate inverse quantized transformcoefficients. For example, operation 1406 may be applied via adaptiveinverse quantize module 203.

Process 1400 may continue at operation 1408, “On each Decoded Block ofCoefficients in a Coding (or Intra Predicted) Partition Perform InverseTransform based on Transform Type and Block Size Data to GenerateDecoded Prediction Error Partitions”, where, on each decode block oftransform coefficients in a coding (or intra predicted) partition, aninverse transform based on the transform type and block size data may beperformed to generate decoded prediction error partitions. In someexamples, the inverse transform may include an inverse fixed transform.In some examples, the inverse transform may include an inverse contentadaptive transform. In such examples, performing the inverse contentadaptive transform may include determining basis functions associatedwith the inverse content adaptive transform based on a neighboring blockof decoded video data, as discussed herein. Any forward transform usedfor encoding as discussed herein may be used for decoding using anassociated inverse transform. In some examples, the inverse transformmay be performed by adaptive inverse transform module 204. In someexamples, generating the decoded prediction error partitions may alsoinclude assembling coding partitions via coding partitions assembler205.

Process 1400 may continue at operation 1423 “Apply DD/DB Filter,Reconstruct Pixel Data, Assemble into a Picture”, where deblockfiltering (e.g., DD or DB filters) may be applied, pixel data may bereconstructed, and assembled into a picture. For example, after inversescan, inverse transform, and assembling coding partitions, theprediction error data partitions may be added with a predictionpartition to generate reconstructed prediction partitions, which may beassembled into tiles or super-fragments. The assembled tiles orsuper-fragments may be optionally processed via deblock filtering.

Process 1400 may continue at operation 1424 “Apply QR/LF Filter Save inReference Picture Buffers”, where quality restoration filtering (e.g.,QR or LF filtering) may be applied, and the assembled picture may besaved in reference picture buffers. For example, in addition to or inthe alternative to the DD/DB filtering, the assembled tiles orsuper-fragments may be optionally processed via quality restorationfiltering and assembled to generate a picture. The picture may be savedin decoded picture buffer 119 as a reference picture for prediction ofother (e.g., following) pictures.

Process 1400 may continue at operation 1425, “Apply AP/AM Filter, UseDecoded Modifying Characteristics (e.g., mop, syp) to Generate ModifiedReferences for Prediction and Use Motion Vectors and Reference Info,Predicted Partition Info, and Modified References to Generate PredictedPartition”, where modified references for prediction may be generatedand predicted partitions may be generated as well, and where adaptivemotion filtering or adaptive precision filtering (e.g., AP/AM Filter)may be applied. For example, where modified references for predictionmay be generated based at least in part on decoded modifyingcharacteristics (e.g., mop, syp) and predicted partitions may begenerated based at least in part on motion vectors and referenceinformation, predicted partition information, and modified references.Additionally, adaptive motion filtering or adaptive precision filteringmay be applied at this point in the process.

Process 1400 may continue at operation 1429 “Optionally Apply EP Filterand/or Optionally apply FI/FP Filter”, where enhanced predictedpartition (e.g., EP Filtering) or FI/FP Filtering (e.g., fusionfiltering or fusion improvement filtering) may be optionally applied. Insome examples, a decision may be made regarding whether to utilize someform or FI/FP Filter (fusion improvement filtering/fusion filtering) ornot to use FI/FP Filtering. When some form or FI/FP Filter (e.g., fusionfiltering or fusion improvement filtering) is to be applied to theselected predicted partition the selected predicted partition and asecond selected predicted partition may be assembled to generate atleast a portion of an assembled picture. FI/FP Filtering may be appliedto filter the portion of the assembled picture. FI/FP Filteringparameters (e.g., filtering parameters or fusion improvement filteringparameters) associated with the FI/FP Filtering may be generated andsent to the entropy coder subsystem.

In implementations where both EP Filtering or FI/FP Filtering areavailable, an indicator may be received from the encoder system thatindicates to the decoder system whether to use the enhanced predictedpartition (e.g., EP Filtering) or the predicted partition data as theselected predicted partition for the prediction partition.

Process 1400 may continue at operation 1430, “Add Prediction Partitionto the Decoded Prediction Error Data Partition to Generate aReconstructed Partition”, where a prediction partition my be added tothe decoded prediction error data partition to generate a reconstructedprediction partition. For example, the decoded prediction error datapartition may be added to the associated prediction partition via adder206.

Process 1400 may continue at operation 1432, “Assemble ReconstructedPartitions to Generate a Tile or Super-Fragment”, where thereconstructed prediction partitions may be assembled to generate tilesor super-fragments. For example, the reconstructed prediction partitionsmay be assembled to generate tiles or super-fragments via predictionpartitions assembler module 207.

Process 1400 may continue at operation 1434, “Assemble Tiles orSuper-Fragments of a Picture to Generate a Full Decoded Picture”, wherethe tiles or super-fragments of a picture may be assembled to generate afull decoded picture. For example, after optional deblock filteringand/or quality restoration filtering, tiles or super-fragments may beassembled to generate a full decoded picture, which may be stored viadecoded picture buffer 210 and/or transmitted for presentment via adisplay device after processing via adaptive picture re-organizer module217 and content post-restorer module 218.

FIGS. 15(A), 15(B), and 15(C) provide an illustrative diagram of anexample video coding system 1600 and video coding process 1500 inoperation, arranged in accordance with at least some implementations ofthe present disclosure. In the illustrated implementation, process 1500may include one or more operations, functions or actions as illustratedby one or more of actions 1501 through 1580. By way of non-limitingexample, process 1500 will be described herein with reference to examplevideo coding system 1600 including encoder 100 of FIG. 1 and decoder 200of FIG. 2, as is discussed further herein below with respect to FIG. 16.In various examples, process 1500 may be undertaken by a systemincluding both an encoder and decoder or by separate systems with onesystem employing an encoder (and optionally a decoder) and anothersystem employing a decoder (and optionally an encoder). It is alsonoted, as discussed above, that an encoder may include a local decodeloop employing a local decoder as a part of the encoder system.

In the illustrated implementation, video coding system 1600 may includelogic circuitry 1650, the like, and/or combinations thereof. Forexample, logic circuitry 1650 may include encoder system 100 of FIG. 1and/or decoder system 200 of FIG. 2 and may include any modules asdiscussed with respect to any of the encoder systems or subsystemsdescribed herein and/or decoder systems or subsystems described herein.Although video coding system 1600, as shown in FIGS. 15(A)-(C) mayinclude one particular set of blocks or actions associated withparticular modules, these blocks or actions may be associated withdifferent modules than the particular modules illustrated here. Althoughprocess 1500, as illustrated, is directed to encoding and decoding, theconcepts and/or operations described may be applied to encoding and/ordecoding separately, and, more generally, to video coding.

Process 1500 may begin at operation 1501, “Receive Input Video Frames ofa Video Sequence”, where input video frames of a video sequence may bereceived via encoder 100 for example.

Process 1500 may continue at operation 1502, “Associate a Picture Typewith each Video Frame in a Group of Pictures”, where a picture type maybe associated with each video frame in a group of pictures via contentpre-analyzer module 102 for example. For example, the picture type maybe F/B-picture , P-picture, or I-picture, or the like. In some examples,a video sequence may include groups of pictures and the processingdescribed herein (e.g., operations 1503 through 1511) may be performedon a frame or picture of a group of pictures and the processing may berepeated for all frames or pictures of a group and then repeated for allgroups of pictures in a video sequence.

Process 1500 may continue at operation 1503, “Divide a Picture intoTiles and/or Super-fragments and Potential Prediction Partitionings”,where a picture may be divided into tiles or super-fragments andpotential prediction partitions via prediction partitions generator 105for example.

Process 1500 may continue at operation 1504, “For Each PotentialPrediction Partitioning, Perform Prediction(s) and Determine PredictionParameters”, where, for each potential prediction partitionings,prediction(s) may be performed and prediction parameters may bedetermined. For example, a range of potential prediction partitionings(each having various prediction partitions) may be generated and theassociated prediction(s) and prediction parameters may be determined.For example, the prediction(s) may include prediction(s) usingcharacteristics and motion based multi-reference predictions orintra-predictions.

As discussed, in some examples, inter-prediction may be performed. Insome examples, up to 4 decoded past and/or future pictures and severalmorphing/synthesis predictions may be used to generate a large number ofreference types (e.g., reference pictures). For instance in ‘inter’mode, up to 9 reference types may be supported in P-pictures, and up to10 reference types may be supported for F/B-pictures. Further, ‘multi’mode may provide a type of inter prediction mode in which instead of 1reference picture, 2 reference pictures may be used and P- andF/B-pictures respectively may allow 3, and up to 8 reference types. Forexample, prediction may be based on a previously decoded frame generatedusing at least one of a morphing technique or a synthesizing technique.In such examples, and the bitstream (discussed below with respect tooperation 1512) may include a frame reference, morphing parameters, orsynthesizing parameters associated with the prediction partition.

Process 1500 may continue at operation 1505, “For Each PotentialPrediction Partitioning, Determine Potential Prediction Error”, where,for each potential prediction partitioning, a potential prediction errormay be determined. For example, for each prediction partitioning (andassociated prediction partitions, prediction(s), and predictionparameters), a prediction error may be determined. For example,determining the potential prediction error may include differencingoriginal pixels (e.g., original pixel data of a prediction partition)with prediction pixels. In some examples, the associated predictionparameters may be stored. As discussed, in some examples, the predictionerror data partition may include prediction error data generated basedat least in part on a previously decoded frame generated using at leastone of a morphing technique or a synthesizing technique.

Process 1500 may continue at operation 1506, “Select PredictionPartitioning and Prediction Type and Save Parameters”, where aprediction partitioning and prediction type may be selected and theassociated parameters may be saved. In some examples, the potentialprediction partitioning with a minimum prediction error may be selected.In some examples, the potential prediction partitioning may be selectedbased on a rate distortion optimization (RDO).

Process 1500 may continue at operation 1507, “Perform Fixed or ContentAdaptive Transforms with Various Block Sizes on Various Potential CodingPartitionings of Partition Prediction Error Data”, where fixed orcontent adaptive transforms with various block sizes may be performed onvarious potential coding partitionings of partition prediction errordata. For example, partition prediction error data may be partitioned togenerate a plurality of coding partitions. For example, the partitionprediction error data may be partitioned by a bi-tree coding partitionermodule or a k-d tree coding partitioner module of coding partitionsgenerator module 107 as discussed herein. In some examples, partitionprediction error data associated with an F/B- or P-picture may bepartitioned by a bi-tree coding partitioner module. In some examples,video data associated with an I-picture (e.g., tiles or super-fragmentsin some examples) may be partitioned by a k-d tree coding partitionermodule. In some examples, a coding partitioner module may be chosen orselected via a switch or switches. For example, the partitions may begenerated by coding partitions generator module 107.

Process 1500 may continue at operation 1508, “Determine the Best CodingPartitioning, Transform Block Sizes, and Actual Transform”, where thebest coding partitioning, transform block sizes, and actual transformsmay be determined. For example, various coding partitionings (e.g.,having various coding partitions) may be evaluated based on RDO oranother basis to determine a selected coding partitioning (which mayalso include further division of coding partitions into transform blockswhen coding partitions to not match a transform block size asdiscussed). For example, the actual transform (or selected transform)may include any content adaptive transform or fixed transform performedon coding partition or block sizes as described herein.

Process 1500 may continue at operation 1509, “Quantize and ScanTransform Coefficients”, where transform coefficients associated withcoding partitions (and/or transform blocks) may be quantized and scannedin preparation for entropy coding.

Process 1500 may continue at operation 1511, “Entropy Encode Dataassociated with Each Tile or Super-fragment”, where data associated witheach tile or super-fragment may be entropy encoded. For example, dataassociated with each tile or super-fragment of each picture of eachgroup of pictures of each video sequence may be entropy encoded. Theentropy encoded data may include the prediction partitioning, predictionparameters, the selected coding partitioning, the selectedcharacteristics data, motion vector data, quantized transformcoefficients, filter parameters, selection data (such as mode selectiondata), and indictors.

Process 1500 may continue at operation 1512, “Generate Bitstream” wherea bitstream may be generated based on the entropy encoded data. Asdiscussed, in some examples, the bitstream may include a frame orpicture reference, morphing parameters, or synthesizing parametersassociated with a prediction partition.

Process 1500 may continue at operation 1513, “Transmit Bitstream”, wherethe bitstream may be transmitted. For example, video coding system 2400may transmit output bitstream 111, bitstream 2100, or the like via anantenna 2402 (please refer to FIG. 34).

Process 1500 may continue at operation 1520, “Reconstruct Pixel Data,Assemble into a Picture, and Save in Reference Picture Buffers”, wherepixel data may be reconstructed, assembled into a picture, and saved inreference picture buffers. For example, after a local decode loop (e.g.,including inverse scan, inverse transform, and assembling codingpartitions), prediction error data partitions may be generated. Theprediction error data partitions may be added with a predictionpartition to generate reconstructed prediction partitions, which may beassembled into tiles or super-fragments. The assembled tiles orsuper-fragments may be optionally processed via deblock filtering and/orquality restoration filtering and assembled to generate a picture. Thepicture may be saved in decoded picture buffer 119 as a referencepicture for prediction of other (e.g., following) pictures.

Process 1500 may continue at operation 1523 “Apply DD/DB Filter,Reconstruct Pixel Data, Assemble into a Picture”, where deblockfiltering (e.g., DD or DB filters) may be applied, pixel data may bereconstructed, and assembled into a picture. For example, after a localdecode loop (e.g., including inverse scan, inverse transform, andassembling coding partitions), prediction error data partitions may begenerated. The prediction error data partitions may be added with aprediction partition to generate reconstructed prediction partitions,which may be assembled into tiles or super-fragments. The assembledtiles or super-fragments may be optionally processed via deblockfiltering and/or quality restoration filtering and assembled to generatea picture.

Process 1500 may continue at operation 1524 “Apply QR/LF Filter Save inReference Picture Buffers”, where quality restoration filtering (e.g.,QR or LF filtering) may be applied, and the assembled picture may besaved in reference picture buffers. For example, in addition to or inthe alternative to the DD/DB filtering, the assembled tiles orsuper-fragments may be optionally processed via quality restorationfiltering and assembled to generate a picture. The picture may be savedin decoded picture buffer 119 as a reference picture for prediction ofother (e.g., following) pictures.

Process 1500 may continue at operation 1525, “Generate ModifyingCharacteristic Parameters”, where, modified characteristic parametersmay be generated. For example, a second modified prediction referencepicture and second modifying characteristic parameters associated withthe second modified prediction reference picture may be generated basedat least in part on the second decoded prediction reference picture,where the second modified reference picture may be of a different typethan the first modified reference picture.

Process 1500 may continue at operation 1526, “Generate ModifiedPrediction Reference Pictures”, where modified prediction referencepictures may be generated, for example, a first modified predictionreference picture and first modifying characteristic parametersassociated with the first modified prediction reference picture may begenerated based at least in part on the first decoded predictionreference picture.

Process 1500 may continue at operation 1527, “Generate Motion Data”,where, motion estimation data may be generated. For example, motion dataassociated with a prediction partition of a current picture may begenerated based at least in part on one of the first modified predictionreference picture or the second modified prediction reference picture.

Process 1500 may continue at operation 1528, “Apply AP/AM Filter PerformMotion Compensation”, where, motion compensation may be performed. Forexample, motion compensation may be performed based at least in part onthe motion data and at least one of the first modified predictionreference picture or the second modified prediction reference picture togenerate prediction partition data for the prediction partition andadaptive motion filtering or adaptive precision filtering (e.g., AP/AMFilter) may be applied. Process 1500 may feed this information back tooperation 1504 where each decoded prediction error partition (e.g.,including zero prediction error partitions) may be added to thecorresponding prediction partition to generate a reconstructedprediction partition. Additionally, adaptive motion filtering oradaptive precision filtering may be applied at this point in theprocess.

Process 1500 may continue at operation 1529 “Optionally Apply EP”, whereenhanced predicted partition (e.g., EP Filtering) may be optionallyapplied. In some examples, where both EP Filtering or FI/FP Filteringare available, an indicator may be generated that indicates to thedecoder system whether to use the enhanced predicted partition (e.g., EPFiltering) or the predicted partition data as the selected predictedpartition for the prediction partition.

Process 1500 may continue at operation 1530 “Optionally apply FI/FPFilter”, where FI/FP Filtering (e.g., fusion filtering or fusionimprovement filtering) may be optionally applied. In some examples, adecision may be made regarding whether to utilize some form or FI/FPFilter (fusion improvement filtering/fusion filtering) or not to useFI/FP Filtering. When some form or FI/FP Filter (e.g., fusion filteringor fusion improvement filtering) is to be applied to the selectedpredicted partition the selected predicted partition and a secondselected predicted partition may be assembled to generate at least aportion of an assembled picture. FI/FP Filtering may be applied tofilter the portion of the assembled picture. FI/FP Filtering parameters(e.g., filtering parameters or fusion improvement filtering parameters)associated with the FI/FP Filtering may be generated and sent to theentropy coder subsystem.

Operations 1501 through 1530 may provide for video encoding andbitstream transmission techniques, which may be employed by an encodersystem as discussed herein. The following operations, operations 1554through 1568 may provide for video decoding and video displaytechniques, which may be employed by a decoder system as discussedherein.

Process 1500 may continue at operation 1554, “Receive Bitstream”, wherethe bitstream may be received. For example, input bitstream 201,bitstream 2100, or the like may be received via decoder 200. In someexamples, the bitstream may include data associated with a codingpartition, one or more indicators, and/or data defining codingpartition(s) as discussed above. In some examples, the bitstream mayinclude the prediction partitioning, prediction parameters, the selectedcoding partitioning, the selected characteristics data, motion vectordata, quantized transform coefficients, filter parameters, selectiondata (such as mode selection data), and indictors.

Process 1500 may continue at operation 1555, “Decode Bitstream”, wherethe received bitstream may be decoded via adaptive entropy decodermodule 202 for example. For example, received bitstream may be entropydecoded to determine the prediction partitioning, prediction parameters,the selected coding partitioning, the selected characteristics data,motion vector data, quantized transform coefficients, filter parameters,selection data (such as mode selection data), and indictors.

Process 1500 may continue at operation 1556, “Perform Inverse Scan andInverse Quantization on Each Block of Each Coding Partition”, where aninverse scan and inverse quantization may be performed on each block ofeach coding partition for the prediction partition being processed. Forexample, the inverse scan and inverse quantization may be performed viaadaptive inverse quantize module 203.

Process 1500 may continue at operation 1557, “Perform Fixed or ContentAdaptive Inverse Transform to Decode Transform Coefficients to DetermineDecoded Prediction Error Data Partitions”, where a fixed or contentadaptive inverse transform may be performed to decode transformcoefficients to determine decoded prediction error data partitions. Forexample, the inverse transform may include an inverse content adaptivetransform such as a hybrid parametric Haar inverse transform such thatthe hybrid parametric Haar inverse transform may include a parametricHaar inverse transform in a direction of the parametric transformdirection and a discrete cosine inverse transform in a directionorthogonal to the parametric transform direction. In some examples, thefixed inverse transform may include a discrete cosine inverse transformor a discrete cosine inverse transform approximator. For example, thefixed or content adaptive transform may be performed via adaptiveinverse transform module 204. As discussed, the content adaptive inversetransform may be based on other previously decoded data, such as, forexample, decoded neighboring partitions or blocks. In some examples,generating the decoded prediction error data partitions may includeassembling decoded coding partitions via coding partitions assemblermodule 205.

Process 1500 may continue at operation 1558, “Generate Prediction PixelData for Each Prediction Partition”, where prediction pixel data may begenerated for each prediction partition. For example, prediction pixeldata may be generated using the selected prediction type (e.g., based oncharacteristics and motion, or intra-, or other types) and associatedprediction parameters.

Process 1500 may continue at operation 1559, “Add to Each DecodedPrediction Error Partition the Corresponding Prediction Partition toGenerate Reconstructed Prediction Partition”, where each decodedprediction error partition (e.g., including zero prediction errorpartitions) may be added to the corresponding prediction partition togenerated a reconstructed prediction partition. For example, predictionpartitions may be generated via the decode loop illustrated in FIG. 2and added via adder 206 to decoded prediction error partitions.

Process 1500 may continue at operation 1560, “Assemble ReconstructedPrediction Partitions to Generate Decoded Tiles or Super-fragments”,where reconstructed prediction partitions may be assembled to generatedecoded tiles or super-fragments. For example, prediction partitions maybe assembled to generate decoded tiles or super-fragments via predictionpartitions assembler module 207.

Process 1500 may continue at operation 1561, “Apply Deblock Filteringand/or QR Filtering to Generate Final Decoded Tiles or Super-fragments”,where optional deblock filtering and/or quality restoration filteringmay be applied to the decoded tiles or super-fragments to generate finaldecoded tiles or super-fragments. For example, optional deblockfiltering may be applied via deblock filtering module 208 and/oroptional quality restoration filtering may be applied via qualityrestoration filtering module 209.

Process 1500 may continue at operation 1562, “Assemble Decoded Tiles orSuper-fragments to Generate a Decoded Video Picture, and Save inReference Picture Buffers”, where decoded (or final decoded) tiles orsuper-fragments may be assembled to generate a decoded video picture,and the decoded video picture may be saved in reference picture buffers(e.g., decoded picture buffer 210) for use in future prediction.

Process 1500 may continue at operation 1563, “Transmit Decoded VideoFrames for Presentment via a Display Device”, where decoded video framesmay be transmitted for presentment via a display device. For example,decoded video pictures may be further processed via adaptive picturere-organizer 217 and content post restorer module 218 and transmitted toa display device as video frames of display video 219 for presentment toa user. For example, the video frame(s) may be transmitted to a displaydevice 2405 (as shown in FIG. 34) for presentment.

Process 1500 may continue at operation 1573 “Apply DD/DB Filter,Reconstruct Pixel Data, Assemble into a Picture”, where deblockfiltering (e.g., DD or DB filters) may be applied, pixel data may bereconstructed, and assembled into a picture. For example, after inversescan, inverse transform, and assembling coding partitions, theprediction error data partitions may be added with a predictionpartition to generate reconstructed prediction partitions, which may beassembled into tiles or super-fragments. The assembled tiles orsuper-fragments may be optionally processed via deblock filtering.

Process 1500 may continue at operation 1574 “Apply QR/LF Filter Save inReference Picture Buffers”, where quality restoration filtering (e.g.,QR or LF filtering) may be applied, and the assembled picture may besaved in reference picture buffers. For example, in addition to or inthe alternative to the DD/DB filtering, the assembled tiles orsuper-fragments may be optionally processed via quality restorationfiltering and assembled to generate a picture. The picture may be savedin a picture buffer as a reference picture for prediction of other(e.g., following) pictures.

Process 1500 may continue at operation 1576, “Generate ModifiedPrediction Reference Pictures”, where modified prediction referencepictures may be generated, for example, at least a portion of a thirdmodified prediction reference picture may be generated based at least inpart on the third modifying characteristic parameters. Similarly, atleast a portion a fourth modified prediction reference picture may begenerated based at least in part on the second modifying characteristicparameters associated.

Process 1500 may continue at operation 1577, “Generate Motion Data”,where, motion estimation data may be generated. For example, motion dataassociated with a prediction partition of a current picture may begenerated based at least in part on one of the third modified predictionreference picture or the third modified prediction reference picture.

Process 1500 may continue at operation 1578, “Apply AP/AM Filter andPerform Motion Compensation”, where, motion compensation may beperformed and where adaptive motion filtering or adaptive precisionfiltering (e.g., AP/AM Filter) may be applied. For example, motioncompensation may be performed based at least in part on the motion dataand at least one of the third modified prediction reference picture orthe fourth modified prediction reference picture to generate predictionpartition data for the prediction partition. Process 1300 may feed thisinformation back to operation 1559 where each decoded prediction errorpartition (e.g., including zero prediction error partitions) may beadded to the corresponding prediction partition to generate areconstructed prediction partition. Additionally, adaptive motionfiltering or adaptive precision filtering may be applied at this pointin the process.

Process 1500 may continue at operation 1579 “Optionally Apply EPFilter”, where enhanced predicted partition (e.g., EP Filtering) may beoptionally applied. In some examples, where both EP Filtering or FI/FPFiltering are available, an indicator may be received from the encodersystem that indicates to the decoder system whether to use the enhancedpredicted partition (e.g., EP Filtering) or the predicted partition dataas the selected predicted partition for the prediction partition.

Process 1500 may continue at operation 1580 “Optionally apply FI/FPFilter”, where FI/FP Filtering (e.g., fusion filtering or fusionimprovement filtering) may be optionally applied. In some examples, adecision may be made regarding whether to utilize some form or FI/FPFilter (fusion improvement filtering/fusion filtering) or not to useFI/FP Filtering. When some form or FI/FP Filter (e.g., fusion filteringor fusion improvement filtering) is to be applied to the selectedpredicted partition the selected predicted partition and a secondselected predicted partition may be assembled to generate at least aportion of an assembled picture. FI/FP Filtering may be applied tofilter the portion of the assembled picture. FI/FP Filtering parameters(e.g., filtering parameters or fusion improvement filtering parameters)associated with the FI/FP Filtering may be generated and sent to theentropy coder subsystem.

Process 1500 may be implemented via any of the coder systems asdiscussed herein. Further, process 1500 may be repeated either in serialor in parallel on any number of instantiations of video data such asprediction error data partitions, original data partitions, or waveletdata or the like.

While implementation of the example processes herein may include theundertaking of all operations shown in the order illustrated, thepresent disclosure is not limited in this regard and, in variousexamples, implementation of the example processes herein may include theundertaking of only a subset of the operations shown and/or in adifferent order than illustrated.

FIG. 16 is an illustrative diagram of example video coding system 1600,arranged in accordance with at least some implementations of the presentdisclosure. In the illustrated implementation, video coding system 1600may include imaging device(s) 1601, video encoder 100 and/or a videoencoder implemented via logic circuitry 1650 of processing unit(s) 1620,video decoder 200 and/or a video decoder implemented via logic circuitry1650 of processing unit(s) 1620, an antenna 1602, one or moreprocessor(s) 1603, one or more memory store(s) 2004, and/or a displaydevice 1605.

As illustrated, imaging device(s) 1601, antenna 1602, processing unit(s)1620, logic circuitry 1650, video encoder 100, video decoder 200,processor(s) 1603, memory store(s) 1604, and/or display device 1605 maybe capable of communication with one another. As discussed, althoughillustrated with both video encoder 100 and video decoder 200, videocoding system 1600 may include only video encoder 100 or only videodecoder 200 in various examples.

As shown, in some examples, video coding system 1600 may include antenna1602. Antenna 1602 may be configured to transmit or receive an encodedbitstream of video data, for example. Further, in some examples, videocoding system 1600 may include display device 1605. Display device 1605may be configured to present video data. As shown, in some example,logic circuitry 1650 may be implemented via processing unit(s) 1620.Processing unit(s) 1620 may include application-specific integratedcircuit (ASIC) logic, graphics processor(s), general purposeprocessor(s), or the like. Video coding system 1600 also may includeoptional processor(s) 1603, which may similarly includeapplication-specific integrated circuit (ASIC) logic, graphicsprocessor(s), general purpose processor(s), or the like. In someexamples, logic circuitry 1650 may be implemented via hardware or videocoding dedicated hardware or the like, and processor(s) 1603 mayimplemented general purpose software or operating systems or the like.In addition, memory stores 1604 may be any type of memory such asvolatile memory (e.g., Static Random Access Memory (SRAM), DynamicRandom Access Memory (DRAM), etc.) or non-volatile memory (e.g., flashmemory, etc.), and so forth. In a non-limiting example, memory stores1604 may be implemented by cache memory. In some examples, logiccircuitry 1650 may access memory stores 1604 (for implementation of animage buffer for example). In other examples, logic circuitry 1650and/or processing unit(s) 1620 may include memory stores (e.g., cache orthe like) for the implementation of an image buffer or the like.

In some examples, video encoder 100 implemented via logic circuitry mayinclude an image buffer (e.g., via either processing unit(s) 1620 ormemory store(s) 1604)) and a graphics processing unit (e.g., viaprocessing unit(s) 1620). The graphics processing unit may becommunicatively coupled to the image buffer. The graphics processingunit may include video encoder 100 as implemented via logic circuitry1650 to embody the various modules as discussed with respect to FIG. 1and FIGS. 3 and 5. For example, the graphics processing unit may includeentropy encoder logic circuitry, and so on. The logic circuitry may beconfigured to perform the various operations as discussed herein. Forexample, the entropy encoder logic circuitry may be configured toreceive first video data and second video data of different types forentropy encoding, determine a first entropy encoding technique for thefirst video data based at least in part on a parameter associated withthe first video data such that the first entropy encoding techniquecomprises at least one of an adaptive symbol-run variable length codingtechnique or an adaptive proxy variable length coding technique, entropyencode the first video data using the first encoding technique togenerate first compressed video data and the second video data using asecond encoding technique to generate second compressed video data, andassemble the first compressed video data and the second compressed videodata to generate an output bitstream. Video decoder 200 may beimplemented in a similar manner.

In some examples, antenna 1602 of video coding system 1600 may beconfigured to receive an entropy encoded bitstream of video data. Asdiscussed, the bitstream may include two or more compressed video datatypes. Video coding system 1600 may also include video decoder 200coupled to antenna 1602 and configured to decode the encoded bitstream.For example, video decoder 200 may be configured to disassemble theentropy encoded bitstream to determine first compressed video data andsecond compressed video data, determine a first entropy decodingtechnique for the first compressed video data, such that the firstentropy decoding technique comprises at least one of an adaptivesymbol-run variable length coding technique or an adaptive proxyvariable length coding technique, entropy decode the first compressedvideo data based on the first entropy decoding technique to generatefirst video data and the second compressed video data based on a secondentropy decoding technique to generate second video data, and decode thefirst video data and the second video data to generate a video frame.

In embodiments, features described herein may be undertaken in responseto instructions provided by one or more computer program products. Suchprogram products may include signal bearing media providing instructionsthat, when executed by, for example, a processor, may provide thefunctionality described herein. The computer program products may beprovided in any form of one or more machine-readable media. Thus, forexample, a processor including one or more processor core(s) mayundertake one or more features described herein in response to programcode and/or instructions or instruction sets conveyed to the processorby one or more machine-readable media. In general, a machine-readablemedium may convey software in the form of program code and/orinstructions or instruction sets that may cause any of the devicesand/or systems described herein to implement at least portions of thefeatures described herein.

FIG. 17 is an illustrative diagram of an example system 1700, arrangedin accordance with at least some implementations of the presentdisclosure. In various implementations, system 1700 may be a mediasystem although system 1700 is not limited to this context. For example,system 1700 may be incorporated into a personal computer (PC), laptopcomputer, ultra-laptop computer, tablet, touch pad, portable computer,handheld computer, palmtop computer, personal digital assistant (PDA),cellular telephone, combination cellular telephone/PDA, television,smart device (e.g., smart phone, smart tablet or smart television),mobile internet device (MID), messaging device, data communicationdevice, cameras (e.g. point-and-shoot cameras, super-zoom cameras,digital single-lens reflex (DSLR) cameras), and so forth.

In various implementations, system 1700 includes a platform 1702 coupledto a display 1720. Platform 1702 may receive content from a contentdevice such as content services device(s) 1730 or content deliverydevice(s) 1740 or other similar content sources. A navigation controller1750 including one or more navigation features may be used to interactwith, for example, platform 1702 and/or display 1720. Each of thesecomponents is described in greater detail below.

In various implementations, platform 1702 may include any combination ofa chipset 1705, processor 1710, memory 1712, antenna 1713, storage 1714,graphics subsystem 1715, applications 1716 and/or radio 1718. Chipset1705 may provide intercommunication among processor 1710, memory 1712,storage 1714, graphics subsystem 1715, applications 1716 and/or radio1718. For example, chipset 1705 may include a storage adapter (notdepicted) capable of providing intercommunication with storage 1714.

Processor 1710 may be implemented as a Complex Instruction Set Computer(CISC) or Reduced Instruction Set Computer (RISC) processors, x86instruction set compatible processors, multi-core, or any othermicroprocessor or central processing unit (CPU). In variousimplementations, processor 1710 may be dual-core processor(s), dual-coremobile processor(s), and so forth.

Memory 1712 may be implemented as a volatile memory device such as, butnot limited to, a Random Access Memory (RAM), Dynamic Random AccessMemory (DRAM), or Static RAM (SRAM).

Storage 1714 may be implemented as a non-volatile storage device suchas, but not limited to, a magnetic disk drive, optical disk drive, tapedrive, an internal storage device, an attached storage device, flashmemory, battery backed-up SDRAM (synchronous DRAM), and/or a networkaccessible storage device. In various implementations, storage 1714 mayinclude technology to increase the storage performance enhancedprotection for valuable digital media when multiple hard drives areincluded, for example.

Graphics subsystem 1715 may perform processing of images such as stillor video for display. Graphics subsystem 1715 may be a graphicsprocessing unit (GPU) or a visual processing unit (VPU), for example. Ananalog or digital interface may be used to communicatively couplegraphics subsystem 1715 and display 1720. For example, the interface maybe any of a High-Definition Multimedia Interface, DisplayPort, wirelessHDMI, and/or wireless HD compliant techniques. Graphics subsystem 1715may be integrated into processor 1710 or chipset 1705. In someimplementations, graphics subsystem 1715 may be a stand-alone devicecommunicatively coupled to chipset 1705.

The graphics and/or video processing techniques described herein may beimplemented in various hardware architectures. For example, graphicsand/or video functionality may be integrated within a chipset.Alternatively, a discrete graphics and/or video processor may be used.As still another implementation, the graphics and/or video functions maybe provided by a general purpose processor, including a multi-coreprocessor. In further embodiments, the functions may be implemented in aconsumer electronics device.

Radio 1718 may include one or more radios capable of transmitting andreceiving signals using various suitable wireless communicationstechniques. Such techniques may involve communications across one ormore wireless networks. Example wireless networks include (but are notlimited to) wireless local area networks (WLANs), wireless personal areanetworks (WPANs), wireless metropolitan area network (WMANs), cellularnetworks, and satellite networks. In communicating across such networks,radio 1718 may operate in accordance with one or more applicablestandards in any version.

In various implementations, display 1720 may include any television typemonitor or display. Display 1720 may include, for example, a computerdisplay screen, touch screen display, video monitor, television-likedevice, and/or a television. Display 1720 may be digital and/or analog.In various implementations, display 1720 may be a holographic display.Also, display 1720 may be a transparent surface that may receive avisual projection. Such projections may convey various forms ofinformation, images, and/or objects. For example, such projections maybe a visual overlay for a mobile augmented reality (MAR) application.Under the control of one or more software applications 1716, platform1702 may display user interface 1722 on display 1720.

In various implementations, content services device(s) 1730 may behosted by any national, international and/or independent service andthus accessible to platform 1702 via the Internet, for example. Contentservices device(s) 1730 may be coupled to platform 1702 and/or todisplay 1720. Platform 1702 and/or content services device(s) 1730 maybe coupled to a network 1760 to communicate (e.g., send and/or receive)media information to and from network 1760. Content delivery device(s)1740 also may be coupled to platform 1702 and/or to display 1720.

In various implementations, content services device(s) 1730 may includea cable television box, personal computer, network, telephone, Internetenabled devices or appliance capable of delivering digital informationand/or content, and any other similar device capable of unidirectionallyor bidirectionally communicating content between content providers andplatform 1702 and/display 1720, via network 1760 or directly. It will beappreciated that the content may be communicated unidirectionally and/orbidirectionally to and from any one of the components in system 1700 anda content provider via network 1760. Examples of content may include anymedia information including, for example, video, music, medical andgaming information, and so forth.

Content services device(s) 1730 may receive content such as cabletelevision programming including media information, digital information,and/or other content. Examples of content providers may include anycable or satellite television or radio or Internet content providers.The provided examples are not meant to limit implementations inaccordance with the present disclosure in any way.

In various implementations, platform 1702 may receive control signalsfrom navigation controller 1750 having one or more navigation features.The navigation features of controller 1750 may be used to interact withuser interface 1722, for example. In various embodiments, navigationcontroller 1750 may be a pointing device that may be a computer hardwarecomponent (specifically, a human interface device) that allows a user toinput spatial (e.g., continuous and multi-dimensional) data into acomputer. Many systems such as graphical user interfaces (GUI), andtelevisions and monitors allow the user to control and provide data tothe computer or television using physical gestures.

Movements of the navigation features of controller 1750 may bereplicated on a display (e.g., display 1720) by movements of a pointer,cursor, focus ring, or other visual indicators displayed on the display.For example, under the control of software applications 1716, thenavigation features located on navigation controller 1750 may be mappedto virtual navigation features displayed on user interface 1722, forexample. In various embodiments, controller 1750 may not be a separatecomponent but may be integrated into platform 1702 and/or display 1720.The present disclosure, however, is not limited to the elements or inthe context shown or described herein.

In various implementations, drivers (not shown) may include technologyto enable users to instantly turn on and off platform 1702 like atelevision with the touch of a button after initial boot-up, whenenabled, for example. Program logic may allow platform 1702 to streamcontent to media adaptors or other content services device(s) 1730 orcontent delivery device(s) 1740 even when the platform is turned “off.”In addition, chipset 1705 may include hardware and/or software supportfor 5.1 surround sound audio and/or high definition 7.1 surround soundaudio, for example. Drivers may include a graphics driver for integratedgraphics platforms. In various embodiments, the graphics driver maycomprise a peripheral component interconnect (PCI) Express graphicscard.

In various implementations, any one or more of the components shown insystem 1700 may be integrated. For example, platform 1702 and contentservices device(s) 1730 may be integrated, or platform 1702 and contentdelivery device(s) 1740 may be integrated, or platform 1702, contentservices device(s) 1730, and content delivery device(s) 1740 may beintegrated, for example. In various embodiments, platform 1702 anddisplay 1720 may be an integrated unit. Display 1720 and content servicedevice(s) 1730 may be integrated, or display 1720 and content deliverydevice(s) 1740 may be integrated, for example. These examples are notmeant to limit the present disclosure.

In various embodiments, system 1700 may be implemented as a wirelesssystem, a wired system, or a combination of both. When implemented as awireless system, system 1700 may include components and interfacessuitable for communicating over a wireless shared media, such as one ormore antennas, transmitters, receivers, transceivers, amplifiers,filters, control logic, and so forth. An example of wireless sharedmedia may include portions of a wireless spectrum, such as the RFspectrum and so forth. When implemented as a wired system, system 1700may include components and interfaces suitable for communicating overwired communications media, such as input/output (I/O) adapters,physical connectors to connect the I/O adapter with a correspondingwired communications medium, a network interface card (NIC), disccontroller, video controller, audio controller, and the like. Examplesof wired communications media may include a wire, cable, metal leads,printed circuit board (PCB), backplane, switch fabric, semiconductormaterial, twisted-pair wire, co-axial cable, fiber optics, and so forth.

Platform 1702 may establish one or more logical or physical channels tocommunicate information. The information may include media informationand control information. Media information may refer to any datarepresenting content meant for a user. Examples of content may include,for example, data from a voice conversation, videoconference, streamingvideo, electronic mail (“email”) message, voice mail message,alphanumeric symbols, graphics, image, video, text and so forth. Datafrom a voice conversation may be, for example, speech information,silence periods, background noise, comfort noise, tones and so forth.Control information may refer to any data representing commands,instructions or control words meant for an automated system. Forexample, control information may be used to route media informationthrough a system, or instruct a node to process the media information ina predetermined manner. The embodiments, however, are not limited to theelements or in the context shown or described in FIG. 17.

As described above, system 1700 may be embodied in varying physicalstyles or form factors. FIG. 18 illustrates implementations of a smallform factor device 1800 in which system 1800 may be embodied. In variousembodiments, for example, device 1800 may be implemented as a mobilecomputing device a having wireless capabilities. A mobile computingdevice may refer to any device having a processing system and a mobilepower source or supply, such as one or more batteries, for example.

As described above, examples of a mobile computing device may include apersonal computer (PC), laptop computer, ultra-laptop computer, tablet,touch pad, portable computer, handheld computer, palmtop computer,personal digital assistant (PDA), cellular telephone, combinationcellular telephone/PDA, television, smart device (e.g., smart phone,smart tablet or smart television), mobile internet device (MID),messaging device, data communication device, cameras (e.g.point-and-shoot cameras, super-zoom cameras, digital single-lens reflex(DSLR) cameras), and so forth.

Examples of a mobile computing device also may include computers thatare arranged to be worn by a person, such as a wrist computer, fingercomputer, ring computer, eyeglass computer, belt-clip computer, arm-bandcomputer, shoe computers, clothing computers, and other wearablecomputers. In various embodiments, for example, a mobile computingdevice may be implemented as a smart phone capable of executing computerapplications, as well as voice communications and/or datacommunications. Although some embodiments may be described with a mobilecomputing device implemented as a smart phone by way of example, it maybe appreciated that other embodiments may be implemented using otherwireless mobile computing devices as well. The embodiments are notlimited in this context.

As shown in FIG. 18, device 1800 may include a housing 1802, a display1804, an input/output (I/O) device 1806, and an antenna 1808. Device1800 also may include navigation features 1812. Display 1804 may includeany suitable display unit for displaying information appropriate for amobile computing device. I/O device 1806 may include any suitable I/Odevice for entering information into a mobile computing device. Examplesfor I/O device 1806 may include an alphanumeric keyboard, a numerickeypad, a touch pad, input keys, buttons, switches, rocker switches,microphones, speakers, voice recognition device and software, and soforth. Information also may be entered into device 1800 by way ofmicrophone (not shown). Such information may be digitized by a voicerecognition device (not shown). The embodiments are not limited in thiscontext.

Various embodiments may be implemented using hardware elements, softwareelements, or a combination of both. Examples of hardware elements mayinclude processors, microprocessors, circuits, circuit elements (e.g.,transistors, resistors, capacitors, inductors, and so forth), integratedcircuits, application specific integrated circuits (ASIC), programmablelogic devices (PLD), digital signal processors (DSP), field programmablegate array (FPGA), logic gates, registers, semiconductor device, chips,microchips, chip sets, and so forth. Examples of software may includesoftware components, programs, applications, computer programs,application programs, system programs, machine programs, operatingsystem software, middleware, firmware, software modules, routines,subroutines, functions, methods, procedures, software interfaces,application program interfaces (API), instruction sets, computing code,computer code, code segments, computer code segments, words, values,symbols, or any combination thereof. Determining whether an embodimentis implemented using hardware elements and/or software elements may varyin accordance with any number of factors, such as desired computationalrate, power levels, heat tolerances, processing cycle budget, input datarates, output data rates, memory resources, data bus speeds and otherdesign or performance constraints.

One or more aspects of at least one embodiment may be implemented byrepresentative instructions stored on a machine-readable medium whichrepresents various logic within the processor, which when read by amachine causes the machine to fabricate logic to perform the techniquesdescribed herein. Such representations, known as “IP cores” may bestored on a tangible, machine readable medium and supplied to variouscustomers or manufacturing facilities to load into the fabricationmachines that actually make the logic or processor.

As discussed, systems, apparatus, articles, and methods are describedherein related to content adaptive entropy coding for video systems. Insome examples, systems, apparatus, articles, and methods are describedherein related to content adaptive prediction entropy coding for motionvectors (e.g., difference motion vectors) for video systems.

In some examples, a selected motion vector prediction technique may bedetermined for a block of video data from a plurality of motion vectorprediction techniques. A prediction motion vector may be generated forthe block of video data based at least in part on the selected motionvector prediction technique. The prediction motion vector may bedifferenced with an original motion vector associated with the block ofvideo data to generate a motion vector differential. The motion vectordifferential may be entropy to generate coded motion vector data. Thecoded motion vector data may be assembled with other types of coded datato generate an output bitstream, which may be transmitted to a decodersystem as discussed herein. The output bitstream (or an encodedbitstream) may be received by a video decoder system. The video decodersystem may disassemble the bitstream to determine coded video data anddecode the coded video data to determine a difference motion vectorassociated with a block of video data. A selected motion vectorprediction technique for the block of video data may be determined froma plurality of motion vector prediction techniques and a predictionmotion vector for the block of video data may be generated based atleast in part on the selected motion vector prediction technique. Thedifference motion vector and the predicted motion vector may be added togenerate a reconstructed motion vector for the block of video data(e.g., the reconstructed motion vector may match the original motionvector) and a video frame may be generated for eventual display to auser based on the reconstructed motion vector and other decodingtechniques as discussed herein.

In some examples, determination of a prediction motion vector mayinclude a spiral motion vector prediction technique. For example, apattern may be determined to provide a sequence of neighboring blocks ofa block of video data such that the pattern has a substantially spiralshape. Original motion vectors associated with the neighboring blocksmay be scanned according to the sequence for a first non-zero originalmotion vector. When a first non-zero original motion vector isdetermined, a prediction motion vector associated with the block ofvideo data may be generated as the first non-zero original motionvector. And when no non-zero original is determined, a prediction motionvector associated with the block of video data may be generated as azero vector.

As discussed herein, encoder module 406 may include motion vectorpredictor module 661, adaptive VLC encoder for motion vector differencesmodule 662, and/or a differencer 663 (please refer to FIG. 6). Also asdiscussed, encoder module 406 may receive video data 416, which mayinclude motion vector data, via motion vector predictor module 661.Motion vector predictor module 661 may perform motion vector predictionbased on video data 416 (e.g., the motion vector data of video data 416)using original motion vector(s) therein to generate associated predictedmotion vector(s). Differencer 663 may difference the predicted motionvector(s) and the original motion vector(s) to generate difference(e.g., differential) motion vector(s) for entropy coding. Adaptive VLCencoder for motion vector differences module 662 may apply an adaptivevariable length coding technique to the difference motion vector(s) togenerate compressed video data 426.

FIG. 19 is an illustrative diagram of an example motion vector anddifferential motion vector encoder module 406, arranged in accordancewith at least some implementations of the present disclosure. As shown,motion vector and differential motion vector encoder module 406, whichmay be described as encoder module 406 herein for the sake of brevity,may be implemented via adaptive entropy encoder 110. The other modules(e.g., encoder modules 401-405 and 407) of adaptive entropy encoder 110may be implemented as described elsewhere herein and will not bedescribed further with respect to FIG. 19. In some examples, motionvector and differential motion vector encoder module 406 may provideefficiency by incorporating a number of adaptive techniques. Further,motion vector and differential motion vector encoder module 406 mayallow coding efficiency while providing substantially low complexity.

Also as illustrated in FIG. 19, vector and differential motion vectorencoder module 406 may include motion vector predictor module 661,adaptive VLC encoder for motion vector differences module 662, and/or adifferencer 663. Adaptive motion predictor module may include adaptivemotion prediction analyzer module 1901 and/or adaptive motion vectorprediction module 1902. As shown, video data 416 may include motionvector data (e.g., original motion vector data, labeled “mv”), which maybe received by motion vector predictor module 661 via adaptive motionprediction analyzer module 1901 and/or adaptive motion vector predictionmodule 1902 for encoding. In some examples, video data 416 may includethe block of video data for which a motion vector is to be encoded,neighboring original motion vectors associated with neighboring blocksof the block of video data, or the like. As shown, encoder module 406may provide compressed video data 426. For example, adaptive VLC encoderfor motion vector differences module 662 may process motion vectordifferentials (e.g., labeled “mvd”) to provide compressed video data 426including a coded representation of the motion vector differentials(e.g., labeled “coded mvd”). As discussed herein, compressed video data426 may be provided to bitstream assembler 408, which may multiplexcompressed video data 426 with compressed video data 421-425 and 427 andprovide output bitstream 111.

As discussed, in some examples, motion vector and differential motionvector encoder module 406 may provide efficiency by incorporating anumber of adaptive techniques. In some examples, adaptive motionprediction analyzer module 1901 may determine, from multiple motionvector prediction techniques, a selected motion vector predictiontechnique for a block of video data (e.g., a current block of video datafor which a prediction motion is being determined). Adaptive motionvector prediction module 1902 may generate a prediction motion vectorfor the block of video data based on the selected motion vectorprediction technique, for example. In some examples, differencer 663 maydifference the prediction motion vector with an original motion vectorassociated with the block of video data to generate a motion vectordifferential. The motion vector differential may be entropy encoded togenerate the coded motion vector data of compressed video data 426, forexample.

In some examples, the multiple motion vector prediction techniques (fromwhich selected motion vector prediction techniques may be chosen forvideo blocks) may include one or more of a median motion vectorprediction technique, a Graham motion vector prediction technique, aGraham last block motion vector prediction technique, or a spiral motionvector prediction technique, or the like.

FIG. 20 is an illustrative diagram of an example video block 2010(labeled current block 2010) and neighboring blocks 2020-2060, arrangedin accordance with at least some implementations of the presentdisclosure. In some examples, current block 2010 may be a block of videodata for which a prediction motion vector is to be determined. As shown,in some examples, immediate neighboring blocks to current block 2010 maybe used to determine a prediction motion vector for current block 2010.For example, neighboring block TL 2020 (e.g., top left) may be to theleft of and above current block 2010. Similarly, neighboring blockT_first 2030 (e.g., top first) may be above and aligned with a left sideof current block 2010, neighboring block T_last 2040 (e.g., top last)may be above and aligned with a right side of current block 2010,neighboring block L_first 2050 (e.g., left first) may be to the left ofand aligned with a top side of current block 2010, and/or neighboringblock L_last 2060 (e.g., left last) may be to the left of and alignedwith a bottom side of current block 2010. Similarly, other immediatelyneighboring or neighboring but removed from (e.g., not in contact with)current block 2010 may be defined.

FIG. 21 is an illustrative diagram of an example video block 2010 andneighboring blocks 2120-2140 for use in example motion vector predictiontechniques, arranged in accordance with at least some implementations ofthe present disclosure. As shown, in some examples, neighboring block2120 (e.g., with x,y-components labeled Ax, Ay) may be above and alignedwith a left side of current block 2010, neighboring block 2130 (e.g.,with x,y-components labeled Bx, By) may be to the left of and abovecurrent block 2010, and neighboring block 2140 (e.g., withx,y-components labeled Cx, Cy) may be to the left of and aligned with atop side of current block 2010. In some examples, neighboring block2120,2130, and 2140 may be analogous to neighboring blocks T first 2030,TL 2020, and L first 2050 of FIG. 20. As shown in FIG. 21, neighboringblocks 2120-2140 may have any size or shape. Similarly, current block2010 may have any size or shape.

As discussed, a selected motion vector prediction technique may bedetermined for current block 2010 from multiple motion vector predictiontechniques. In some examples, the multiple motion vector predictiontechniques may include a median motion vector prediction technique. Withreference to FIG. 20, a median motion vector prediction technique mayprovide a prediction motion vector (Px, Py) for current block 2010 basedon neighboring block 2120, 2130, and 2140 as follows. For example, thefollowing may be determined from video data 316:

-   Ax=x coordinate of motion vector of block T first (e.g., neighboring    block 2120)-   Ay=y coordinate of motion vector of block T first (e.g., neighboring    block 2120)-   Bx=x coordinate of motion vector of block TL (e.g., neighboring    block 2130)-   By=y coordinate of motion vector of block TL (e.g., neighboring    block 2130)-   Cx=x coordinate of motion vector of block L first (e.g., neighboring    block 2140)-   Cy=y coordinate of motion vector of block L first (e.g., neighboring    block 2140)

In some examples, the prediction motion vector (Px, Py) for currentblock 2010 via the median motion vector prediction technique may be asfollows:

-   Px=median(Ax, Bx , Cx);-   Py=median(Ay, By , Cy).

For example, the median motion vector prediction technique may includedetermining a first neighboring original motion vector for a firstneighboring block of video data above and aligned with a left edge ofthe block of video data (e.g., Ax, Ay for neighboring block 2120), asecond neighboring original motion vector for a second neighboring blockof video data to the left of and above the block of video data (e.g.,Bx, By for neighboring block 2130), and a third neighboring originalmotion vector for a third neighboring block of video data to the left ofand aligned with a top edge of the block of video data (e.g., Cx, Cy forneighboring block 2140), and generating the prediction motion vector forthe block of video data as the median of the first, second and thirdneighboring original motion vectors.

In some examples, the multiple motion vector prediction techniques mayinclude a Graham motion vector prediction technique. With reference toFIG. 20, a Graham motion vector prediction technique may provide aprediction motion vector (Px, Py) for current block 2010 based onneighboring block 2120, 2130, and 2140 as follows. For example, thefollowing may be determined from video data 316:

-   Ax=x coordinate of motion vector of block T_first (e.g., neighboring    block 2120)-   Ay=y coordinate of motion vector of block T_first (e.g., neighboring    block 2120)-   Bx=x coordinate of motion vector of block TL (e.g., neighboring    block 2130)-   By=y coordinate of motion vector of block TL (e.g., neighboring    block 2130)-   Cx=x coordinate of motion vector of block L_first (e.g., neighboring    block 2140)-   Cy=y coordinate of motion vector of block L_first (e.g., neighboring    block 2140)

In some examples, the following absolute values may be determined:

DXx=abs(Cx-Bx)

DXy=abs(Cy-By)

DYx=abs(Ax-Bx)

DYy=abs(Ay-By)

where abs is a function that takes the absolute value of the givenvalue.

In some examples, the prediction motion vector (Px, Py) for currentblock 2010 via the Graham motion vector prediction technique may be asfollows:

${Px} = \left\{ {{\begin{matrix}{{Ax},} & {{{{if}\mspace{14mu}{DXx}} \leq {DXy}},{or}} \\{{Cx},} & {otherwise}\end{matrix}{Py}} = \left\{ \begin{matrix}{{Ay},} & {{{{if}\mspace{14mu}{DYx}} \leq {DYy}},{or}} \\{{Cy},} & {otherwise}\end{matrix} \right.} \right.$

For example, the Graham motion vector prediction technique may includedetermining a first neighboring original motion vector for a firstneighboring block of video data above and aligned with a left edge ofthe block of video data (e.g., Ax, Ay for neighboring block 2120), asecond neighboring original motion vector for a second neighboring blockof video data to the left of and above the block of video data (e.g.,Bx, By for neighboring block 2130), and a third neighboring originalmotion vector for a third neighboring block of video data to the left ofand aligned with a top edge of the block of video data (e.g., Cx, Cy forneighboring block 2140), determining an absolute value differencebetween the second and the third neighboring original motion vectors(e.g., DXx, DXy), determining an absolute value difference between thefirst and the second neighboring original motion vectors (e.g., DYx,DYy), and generating the prediction motion vector for the block of videodata as the first neighboring original motion vector if the absolutevalue difference between the second and the third neighboring originalmotion vectors is less than or equal to the absolute value differencebetween the first and the second neighboring original motion vectors andas the third neighboring original motion vector if the absolute valuedifference between the second and the third neighboring original motionvectors is greater than the absolute value difference between the firstand the second neighboring original motion vectors (e.g., as shownabove).

FIG. 22 is an illustrative diagram of an example video block 2010 andneighboring blocks 2220-2240 for use in example motion vector predictiontechniques, arranged in accordance with at least some implementations ofthe present disclosure. As shown, in some examples, neighboring block2220 (e.g., with x,y-components labeled Ax, Ay) may be above and alignedwith a right side of current block 2010, neighboring block 2230 (e.g.,with x,y-components labeled Bx, By) may be to the left of and abovecurrent block 2010, and neighboring block 2240 (e.g., withx,y-components labeled Cx, Cy) may be to the left of and aligned with abottom side of current block 2010. In some examples, neighboring block2220,2230, and 2240 may be analogous to neighboring blocks T_last 2060,TL 2020, and L_last 2040 of FIG. 20. As shown in FIG. 22, neighboringblocks 2220-2240 may have any size or shape.

As discussed, a selected motion vector prediction technique may bedetermined for current block 2010 from multiple motion vector predictiontechniques. In some examples, the multiple motion vector predictiontechniques may include a Graham last block motion vector predictiontechnique. With reference to FIG. 22, a Graham last block motion vectorprediction technique may provide a prediction motion vector (Px, Py) forcurrent block 2010 based on neighboring block 2220, 2230, and 2240 asfollows. For example, the following may be determined from video data316:

Ax=x coordinate of motion vector of block T_last (e.g., neighboringblock 2220)

Ay=y coordinate of motion vector of block T_last (e.g., neighboringblock 2220)

Bx=x coordinate of motion vector of block TL (e.g., neighboring block2230)

By=y coordinate of motion vector of block TL (e.g., neighboring block2230)

Cx=x coordinate of motion vector of block L_last (e.g., neighboringblock 2240)

Cy=y coordinate of motion vector of block L_last (e.g., neighboringblock 2240)

In some examples, the following absolute values may be determined:

DXx=abs(Cx-Bx)

DXy=abs(Cy-By)

DYx=abs(Ax-Bx)

DYy=abs(Ay-By)

where abs is a function that takes the absolute value of the givenvalue.

In some examples, the prediction motion vector (Px, Py) for currentblock 2010 via the Graham last block motion vector prediction techniquemay be as follows:

${Px} = \left\{ {{\begin{matrix}{{Ax},} & {{{{if}\mspace{14mu}{DXx}} \leq {DXy}},{or}} \\{{Cx},} & {otherwise}\end{matrix}{Py}} = \left\{ \begin{matrix}{{Ay},} & {{{{if}\mspace{14mu}{DYx}} \leq {DYy}},{or}} \\{{Cy},} & {otherwise}\end{matrix} \right.} \right.$

In some examples, the Graham last block motion vector predictiontechnique may include determining a first neighboring original motionvector for a first neighboring block of video data above and alignedwith a right edge of the block of video data (e.g,. Ax, Ay forneighboring block 2220), a second neighboring original motion vector fora second neighboring block of video data to the left of and above theblock of video data (e.g,. Bx, By for neighboring block 2230), and athird neighboring original motion vector for a third neighboring blockof video data to the left of and aligned with a bottom edge of the blockof video data (e.g,. Cx, Cy for neighboring block 2240), determining anabsolute value difference between the second and the third neighboringoriginal motion vectors (e.g., DXx, DXy), determining an absolute valuedifference between the first and the second neighboring original motionvectors (e.g., DXx, DXy), and generating the prediction motion vectorfor the block of video data as the first neighboring original motionvector if the absolute value difference between the second and the thirdneighboring original motion vectors is less than or equal to theabsolute value difference between the first and the second neighboringoriginal motion vectors and as the third neighboring original motionvector if the absolute value difference between the second and the thirdneighboring original motion vectors is greater than the absolute valuedifference between the first and the second neighboring original motionvectors (e.g., as shown above).

FIG. 23 is an illustrative diagram of an example video block 2010 andneighboring blocks 2320, 2330 for use in example motion vectorprediction techniques, arranged in accordance with at least someimplementations of the present disclosure. As shown, in some examples,neighboring blocks 2320 (some of which are labeled 0-36) and 2330 (oneof which is labeled 37) may surround video block 2010 such that some areimmediately adjacent to video block 2010 (e.g., 0-11) and others areadjacent but not immediately adjacent to video block 2010 (e.g., 12-37and so on). In the example of FIG. 23, neighboring blocks having dashedboundaries may have zero motion vectors and neighboring blocks havingsolid boundaries may have non-zero motion vectors.

As discussed, a selected motion vector prediction technique may bedetermined for current block 2010 from multiple motion vector predictiontechniques. In some examples, the multiple motion vector predictiontechniques may include a spiral motion vector prediction technique. Withreference to FIG. 23, a spiral motion vector prediction technique mayprovide a prediction motion vector for current block 2010 based onneighboring block 2320, 2330 2240 by determining a pattern to provide asequence of neighboring blocks 2320, 2330 of video block 2010 (e.g., ablock of video data) such that the pattern may include a substantiallyspiral shape. In some examples, original motion vectors associated withneighboring blocks 2320, 2330 may be scanned according to the sequence(e.g., as determined by the pattern) for a first (e.g., sequentially)non-zero original motion vector. When a non-zero original motion vectoris determined, the prediction motion vector associated with video block2010 may be generated as the first non-zero original motion vector. Ininstances when no non-zero original motion vector is determined beforethe end of the sequence, the prediction motion vector associated withvideo block 2010 may be a zero vector.

In some examples, the pattern may be defined by the sequence from 0 to37 as shown in FIG. 23. For example, the pattern may include a shapebeginning at a first neighboring block to the left of and aligned withthe top of the block of video data (e.g., neighboring block 0) andprogressing downwards to a second neighboring block to the left of andaligned with the bottom of the block of video data (e.g., neighboringblock 1), then skipping to a third neighboring block above and alignedwith the left of the block of video data (e.g., neighboring block 2),then progressing right to a fourth neighboring block above and alignedwith the right side of the block of video data (e.g., neighboring block3), then skipping to a fifth neighboring block to the left of above theblock of video data (e.g., neighboring block 4), then skipping to asixth neighboring block to the right of and above the block of videodata (e.g., neighboring block 5), then skipping to a seventh neighboringblock to the left of and below the block of video data (e.g.,neighboring block 6), then skipping to an eighth neighboring block tothe right of and below the block of video data (e.g., neighboring block7), then skipping to a ninth neighboring block to the right of andaligned with the top of the block of video data (e.g., neighboring block8), then progressing downward to a tenth neighboring block to the rightof and aligned with the bottom of the block of video data (e.g.,neighboring block 9), and so on. Such a pattern may not be strictlyspiral but may be a substantially spiral as used herein as the patternbegins immediately adjacent to video block 2010, scans the immediatelyadjacent neighboring blocks, continues to scan neighboring blocks onelevel adjacent to video block 2010, and so on.

As shown in FIG. 23, in some examples, the number of scanned neighboringblocks may be 37 blocks or more. In some examples, the number of scannedneighboring blocks may be 20. In some examples, the number of scannedneighboring blocks may be 10, 15, or 25, or the like. In some examples,the scanned neighboring blocks may have any shape or size such assquare, or rectangular or the like. In some examples, the number ofscanned neighboring blocks may be square in shape and have a size of4×4.

In some examples, the pattern may be defined by a sequence as follows(please refer to FIG. 23:0-4-2-3-5-8-9-7-11-10-6-1-13-12-20-15-17-18-19-21-24-25-26-27-23-31-30-39-38-22-15-14-13-12-20-27,and so on. For example, the pattern may include a shape beginning at afirst neighboring block to the left of and aligned with the top of theblock of video data (e.g., neighboring block 0) and progressing upwardsto a second block to the left of and above the block of video data(e.g., neighboring block 4), then to the right along the top of theblock of video data to a third neighboring block to the right of andabove the block of video data (e.g., neighboring block 5), thendownwards to a fourth neighboring block to the right of and below theblock of video data (e.g., neighboring block 7), then left to a fifthneighboring block to the left of and below the block of video data(e.g., neighboring block 6), and so on. In a similar manner, manypatterns may be determined to provide a sequence of neighboring blocks.

FIG. 24 is a flow diagram illustrating an example process 2400, arrangedin accordance with at least some implementations of the presentdisclosure. Process 2400 may include one or more operations, functionsor actions as illustrated by one or more of operations 2402, 2404, 2406,and/or 2408. Process 2400 may form at least part of a next generationvideo coding process. By way of non-limiting example, process 2400 mayform at least part of a next generation video encoding process asundertaken by encoder system 100 of FIG. 1 and/or entropy encoder module110 as discussed herein.

Process 2400 may begin at operation 2402, “Determine a Selected MotionVector Prediction Technique for a Block of Video Data”, where a selectedmotion vector prediction technique may be determined for a block ofvideo data from a plurality of motion vector prediction techniques. Forexample, a selected motion vector prediction may be determined from atleast one of a median motion vector prediction technique, a Grahammotion vector prediction technique, a Graham last block motion vectorprediction technique, or a spiral motion vector prediction technique viaadaptive motion prediction analyzer module 1901.

Process 2400 may continue at operation 2404, “Generate a PredictionMotion Vector for the Block of Video Data Based on the Selected MotionVector Prediction Technique”, where a prediction motion vector may begenerated for the block of video data based at least in part on theselected motion vector prediction technique. For example, adaptivemotion prediction analyzer module 1901 may indicate the selected motionvector prediction technique to adaptive motion vector prediction module1902, which may apply the selected technique to generate a predictionmotion vector.

Process 2400 may continue at operation 2406, “Difference the PredictionMotion Vector with an Original Motion Vector of the Block of VideoData”, where the prediction motion vector may be differenced with anoriginal motion vector associated with the block of video data togenerate a motion vector differential. For example, differencer 663 maydifference the prediction motion vector and original motion vector, andtransmit the resultant motion vector differential (e.g., “mvd”) toadaptive VLC encoder for motion vector differences module 662.

Process 2400 may continue at operation 2408, “Entropy Encode the MotionVector Differential to Generate Coded Motion Vector Data”, where themotion vector differential may be entropy encoded to generate codedmotion vector data. For example, adaptive VLC encoder for motion vectordifferences module 662 may entropy encode the motion vector differentialvia an adaptive variable length coding technique to generated compressedvideo data 426 including coded motion vector data (e.g., “coded mvc”).As is discussed further herein, a switching bit mask may also be encodedvia adaptive VLC encoder for motion vector differences module 662.

As discussed, the coded motion vector data, via compressed video data426, may be assembled with other compressed video data (e.g., compressedvideo data 421-425 and 427) to generate an output bitstream (e.g.,output bitstream 111).

FIG. 25 is a flow diagram illustrating an example process 2500, arrangedin accordance with at least some implementations of the presentdisclosure. Process 2500 may include one or more operations, functionsor actions as illustrated by one or more of operations 2502, 2504, 2506,2508, and/or 2510. Process 2500 may form at least part of a nextgeneration video coding process. By way of non-limiting example, process2500 may form at least part of a next generation video encoding processto determine a prediction motion vector for a current block of videodata as undertaken by encoder system 100 of FIG. 1 and/or entropyencoder module 110 as discussed herein.

Process 2500 may begin at operation 2502, “Determine a SubstantiallySpiral Pattern to Provide a Sequence of Neighboring Blocks”, where apattern may be determined to provide a sequence of neighboring blocks ofa block of video data such that the pattern comprises a substantiallyspiral shape. For example, any shape as discussed herein with respect toFIG. 23 may be determined. For example, the pattern may provide a spiralor a substantially spiral such that the pattern begins immediatelyadjacent to a video block, scans the immediately adjacent neighboringblocks, continues to scan neighboring blocks one level adjacent to thevideo block, and so on. Such a pattern may define a sequence ofneighboring blocks, as discussed herein.

Process 2500 may continue at operation 2504, “Scan Original MotionVectors of the Neighboring Blocks According to the Sequence”, whereoriginal motion vectors associated with the neighboring blocks may bescanned according to the sequence for a first non-zero original motionvector. For example, adaptive motion prediction analyzer module 1901 mayindicate the selected motion vector prediction technique to adaptivemotion vector prediction module 1902, which may apply the selectedtechnique to generate a prediction motion vector.

Process 2500 may continue at decision operation 2506, “Non-Zero MotionVector Determined”, where it may be determined whether a non-zero motionvector is determined prior to the sequence of neighboring blocks ends.If a non-zero motion vector is determined, a prediction motion vectorassociated with the block of video data may be generated as the first(sequentially) non-zero original motion vector, as shown at operation2508, “Generate Motion Vector as First Non-Zero Motion Vector”. If nonon-zero motion vector is determined, a prediction motion vectorassociated with the block of video data may be generated as a zerovector, as shown at operation 2510, “Generate Motion Vector as a ZeroVector”.

As discussed herein, the determined prediction motion vector for theblock of video data may be differenced with an original motion vectorfor the block of video data and the differential may be entropy encoded.

As discussed, a motion vector prediction technique may be determined fora block of video data. In some examples, a predetermined mode or methodmay be established for selection of the motion vector predictiontechnique. For example, in a mode 0 (or method 0), the motion vectorprediction technique may be set to the median motion vector predictiontechnique. In such examples, only the motion vector differential betweenthe predicted motion vector and the original motion vector for a blockof video data may be encoded. The designation of a mode label (e.g.,mode 0 or the like) is for the sake of clarity of presentation and isnot meant to be limiting. For example, one, several, or all of the modesmay be available in various implementations.

In some examples, in a mode 1 (or method 1), the motion vectorprediction technique may switch between the median motion vectorprediction technique and either the Graham motion vector predictiontechnique or the spiral motion vector prediction technique as follows.For example, the median motion vector prediction technique may becomputed first for a block of video data (e.g., a current block). Insuch examples, the median motion vector prediction technique may bedeemed to be a first motion vector prediction technique and a generatedmotion vector may be deemed a first predicted motion vector.

If the resultant motion vector is a zero vector (e.g., zero motiondetermined as 0, 0), then the spiral motion vector prediction techniquemay then be used. In such examples, the spiral motion vector predictiontechnique may be deemed to be a second motion vector predictiontechnique and a generated motion vector may be deemed a second predictedmotion vector.

If the resultant motion vector is a non-zero vector, then the Grahammotion vector prediction technique may then be used. In such examples,the Graham motion vector prediction technique may be deemed to be asecond motion vector prediction technique and a generated motion vectormay be deemed a second predicted motion vector.

In either case, the motion vector determined using the first motionvector prediction technique (e.g., the median motion vector predictiontechnique) may be compared to the motion vector determined using thesecond motion vector prediction technique (e.g., spiral or Graham).

If the two motion vectors are equal or their difference is less than athreshold, then the motion vector prediction technique is determined asthe first motion vector prediction technique (e.g., median).

If the two motion vectors are different, an estimated bit cost for themotion vectors may be determined and the technique (either the first orsecond) providing the lower bit cost may be used. In such examples, abit may be added to a switching bit mask, which may track the motionvector prediction technique for blocks of video data for encoding. Forexample, the switching bit mask may be encoded using bitmap coding,proxy VLC coding, or symbol run coding via adaptive VLC encoder formotion vector differences module 662.

In some examples, a mode 2 (or method 2) may operate the same as mode 1(or method 1), with the exception that in place of the Graham motionvector prediction technique, a Graham last block motion vectorprediction technique may be implemented.

In some examples, a mode 3 (or method 3) may operate the same as mode 1(or method 1), with the exception that if the two determined motionvectors are different but within a threshold, the motion vectordetermined using the second motion vector prediction technique (e.g.,spiral or Graham) may be used (e.g., without determining bit costs). Ifthe two motion vectors differ by more than the threshold, the bit costtechniques as described may be utilized. In some examples, the thresholdmay be predetermined, determined heuristically, or the like.

In some examples, a mode 4 (or method 4) may operate the same as mode 3(or method 3), with the exception that in place of the Graham motionvector prediction technique, a Graham last block motion vectorprediction technique may be implemented.

For example, as discussed, a first prediction motion vector may bedetermined via a first prediction motion vector prediction techniquesuch that the first prediction motion vector prediction technique may bethe median motion vector prediction technique. When the first predictionmotion vector is a zero vector, a second prediction motion vector may bedetermined via a second prediction motion vector prediction techniquesuch that the second prediction motion vector prediction technique maybe the spiral motion vector prediction technique. When the firstprediction motion vector is a non-zero vector, a second predictionmotion vector may be determined via a third prediction motion vectorprediction technique such that the third prediction motion vectorprediction technique may be the Graham motion vector predictiontechnique or the Graham last block motion vector prediction technique(e.g., depending on mode). When a difference between the firstprediction motion vector and the second prediction motion vector is lessthan a threshold, in a first mode, the selected motion vector predictiontechnique may be determined as the first motion vector predictiontechnique and, in a second mode, the selected motion vector predictiontechnique may be determined as the second or third motion vectorprediction techniques (depending further on the mode). When a differencebetween the first prediction motion vector and the second predictionmotion vector is greater than a threshold (in certain modes), a firstestimated bit cost associated with the first motion vector predictiontechnique may be determined and a second estimated bit cost associatedwith the second or third motion vector prediction technique may bedetermined. When the first estimated bit cost is less than the secondestimated bit cost, the selected motion vector prediction technique maybe determined as the first motion vector prediction technique. when thesecond estimated bit cost is less than the first estimated bit cost, theselected motion vector prediction technique may be determined as thesecond or third motion vector prediction technique. As discussed, insome examples, a switching bit may be added to the switching bit maskfor later encoding.

As will be appreciated in mode 0 (or method 0), no switching bit maskmay need to be provided as only median motion vector prediction is used.In other modes, a switching bit mask may be provided on a frame basissuch that the switching bit mask indicates whether a motion compensatedblock uses median motion vector prediction or a selected motion vectorprediction technique. No bit is required to indicated which selectedmotion vector prediction technique (e.g., Graham or spiral in mode 1)because the choice between them is based on the result of the medianmotion vector prediction (e.g., if the first predicted motion vector isa zero motion vector, spiral is used in mode 1 and, if the firstpredicted motion vector is a non-zero motion vector, Graham is used inmode 1). Similarly, modes 1-4 may not require a bit for the selectedmotion vector prediction technique.

The switching cost between the discussed modes (or methods) is depictedthe Table 1. In Table 1, the following abbreviations are used: Hdr(Header), MV (Motion Vector), Pred (Prediction).

TABLE 1 Overview of MV Prediction Methods Set Equal/ Mode/ Hdr 1^(st) MV2^(nd) MV Close Pred Method Hdr Size Summary Prediction Prediction MVs 00 1 No Switching Median N/A N/A mask 1 100 3 Switching MedianGraham/Spiral Yes mask coded 2 101 3 Switching Median Graham Yes maskcoded Last/Spiral 3 110 3 Switching Median Graham/Spiral No mask coded 4111 3 Switching Median Graham No mask coded Last/Spiral

As discussed, motion vector data including motion vectors may beobtained for a video frame. A first predicted motion vector may bedetermined using a median motion vector prediction technique for eachmotion compensated block of the video frame. For modes 1-4, a secondpredicted motion vector may be determined using a selected motion vectorprediction technique for each motion compensated block (the selectedmotion vector prediction technique being based on the mode and theresult of he first prediction motion vector as discussed). The first orthe second predicted motion vector may be selected for each motioncompensated block of the video frame and a motion vector bit mask may bedefined which identifies the median or the selected motion vectorprediction technique (e.g., as associated with the first and secondpredicted motion vectors) for each motion compensated block. As isdiscussed further, a selected coding method may be determined for themotion vector bit mask (e.g., the switching mask) and the motion vectorbit mask may be encoded using the selected coding method into abitstream. The process may be repeated on a frame-by-frame basis.

The bitstream may be received at a decoder and disassembled and/ordecoded to determine a frame level motion vector prediction method(e.g., a mode) and a motion vector bit mask (e.g., a switching mask) fora video frame. As discussed, the motion vector bit mask may identify amedian or a selected motion vector prediction technique for each motioncompensated block of the video frame and the selected motion vectorprediction technique may be based on the frame level motion vectorprediction method and a first motion vector for each motion compensatedblock determined based on the median motion vector prediction technique.For example, for a motion compensated block using the median motionvector prediction technique (as indicated via the bit mask), the decodermay determine the predicted motion vector for the block using the medianmotion vector prediction technique (e.g., for modes 1-4). For a motioncompensated block using the selected motion vector prediction technique(again, as indicated via the bit mask), the decoder may determine afirst predicted motion vector (or a median based prediction motionvector) using the median motion vector prediction technique and a secondpredicted motion vector based on the frame level motion vectorprediction method (e.g., mode) and the result of the median basedprediction motion vector. For example, when the median based predictionmotion vector is a zero motion vector, the second predicted motionvector may be determined using a spiral motion vector predictiontechnique. When the median based prediction motion vector is a non-zeromotion vector, the second predicted motion vector may be determinedusing a Graham motion vector prediction technique (in modes 1 and 3) ora Graham last block motion vector prediction technique (in modes 2 and4). In such a manner, the bitstream may not need an additional bitindicating which selected motion vector prediction technique is to beused for each motion compensated block. Further, as discussed herein,the predicted motion vector may be added to a motion vector differenceto determine a final decoded motion vector for each motion compensatedblock.

FIG. 26 is an illustrative diagram of an example adaptive motionprediction analyzer module 1901, arranged in accordance with at leastsome implementations of the present disclosure. As shown, adaptivemotion prediction analyzer module 1901 may be implemented via motionvector predictor module 661 of entropy encoder 406, which may beimplemented via adaptive entropy encoder 110 as discussed herein. Asshown, adaptive motion prediction analyzer module 1901 may provide aselected motion vector prediction technique to adaptive motion vectorprediction module 1902 (e.g. via “best indx”). Also as shown, adaptivemotion prediction analyzer module 1901 may include a motion vectormedian prediction module 2601, a motion vector Graham/Spiral predictionmodule 2602, a motion vector Graham last (e.g., Graham lastblock)/Spiral prediction module 2603, a motion vector Graham/Spiralprediction module 2604, a motion vector Graham last/Spiral predictionmodule 2605, a selector and switch mask collector modules 2611-2614, apicture/slice switch mask symbol run and/or proxy VLC encoder module2620, and/or a cost estimator and minimum selector module 2630.

In some examples, modules 2601-2605 may implement techniquesimplementing or associated with modes (or methods) 0-4 as discussedabove. For example, motion vector median prediction module 2601 maypredict a motion vector for a block of video data using a median motionvector prediction technique and provide the motion vector to selectorand switch mask collector modules 2611-2614 and picture/slice switchmask symbol run and/or proxy VLC encoder module 2620.

Motion vector Graham/Spiral prediction module 2602 may predict a motionvector for a block of video data using a Graham motion vector predictiontechnique if the median motion vector is a non-zero motion vector or aspiral motion vector prediction technique if the median motion vector isa zero motion vector for implementation of mode 1. As shown in FIG. 26,an input ‘1’ may be provided to motion vector Graham/Spiral predictionmodule 2602 and motion vector Graham last/Spiral prediction module 2603.Also as shown, an input ‘0’ may be provided to motion vectorGraham/Spiral prediction module 2604 and motion vector Grahamlast/Spiral prediction module 2605. For example, such inputs may relateto a set equal close predicted motion vector indicator. For example, anindicator of ‘1’ may provide to set equal close predicted motion vectorswhile an indicator of ‘0’ may provide to not set equal close predictedmotion vectors (please see Table 1). In some examples, such an indicatormay be passed to or provided directly to the appropriate selector andswitch mask collector (e.g., one of selector and switch mask collectors2611-2614).

As shown, selector and switch mask collector 2611 may receive the firstand second predicted motion vectors from motion vector median predictionmodule 2601 (e.g., first) and motion vector Graham/Spiral predictionmodule 2602 (e.g., second), which may collect the motion vectors and aswitch mask for mode 1 as discussed herein. For example, if thedifference between the first and second motion vectors is within athreshold, selector and switch mask collector 2611 may select the firstmotion vector and if the difference is greater than a threshold,selector and switch mask collector 2611 may select both the first andsecond motion vectors for further processing.

Motion vector Graham last/Spiral prediction module 2603 may predict amotion vector for a block of video data using a Graham last block motionvector prediction technique if the median motion vector is a non-zeromotion vector or a spiral motion vector prediction technique if themedian motion vector is a zero motion vector for implementation of mode2. As shown in FIG. 26, an input ‘1’ may be provided to motion vectorGraham last/Spiral prediction module 2603 indicating to set equal closepredicted motion vectors.

As shown, selector and switch mask collector 2612 may receive the firstand second predicted motion vectors from motion vector median predictionmodule 2601 (e.g., first) and motion vector Graham last/Spiralprediction module 2603 (e.g., second), which may collect the motionvectors and a switch mask for mode 2 as discussed herein. For example,if the difference between the first and second motion vectors is withina threshold, selector and switch mask collector 2612 may select thefirst motion vector and if the difference is greater than a threshold,selector and switch mask collector 2612 may select both the first andsecond motion vectors for further processing.

Motion vector Graham/Spiral prediction module 2604 may predict a motionvector for a block of video data using a Graham motion vector predictiontechnique if the median motion vector is a non-zero motion vector or aspiral motion vector prediction technique if the median motion vector isa zero motion vector for implementation of mode 3. As shown in FIG. 26,an input ‘0’ may be provided to motion vector Graham last/Spiralprediction module 2604 indicating to not set equal close predictedmotion vectors.

As shown, selector and switch mask collector 2613 may receive the firstand second predicted motion vectors from motion vector median predictionmodule 2601 (e.g., first) and motion vector Graham/Spiral predictionmodule 2604 (e.g., second), which may collect the motion vectors and aswitch mask for mode 3 as discussed herein. For example, if thedifference between the first and second motion vectors is within athreshold, selector and switch mask collector 2613 may select the secondmotion vector (e.g., not set equal to the first) and if the differenceis greater than a threshold, selector and switch mask collector 2613 mayselect both the first and second motion vectors for further processing.

Motion vector Graham last/Spiral prediction module 2605 may predict amotion vector for a block of video data using a Graham last block motionvector prediction technique if the median motion vector is a non-zeromotion vector or a spiral motion vector prediction technique if themedian motion vector is a zero motion vector for implementation of mode4. As shown in FIG. 26, an input ‘0’ may be provided to motion vectorGraham last/Spiral prediction module 2605 indicating to not set equalclose predicted motion vectors.

As shown, selector and switch mask collector 2614 may receive the firstand second predicted motion vectors from motion vector median predictionmodule 2601 (e.g., first) and motion vector Graham last/Spiralprediction module 2605 (e.g., second), which may collect the motionvectors and a switch mask for mode 4 as discussed herein. For example,if the difference between the first and second motion vectors is withina threshold, selector and switch mask collector 2614 may select thesecond motion vector (e.g., not set equal to the first) and if thedifference is greater than a threshold, selector and switch maskcollector 2614 may select both the first and second motion vectors forfurther processing.

As shown, selector and switch mask collectors 2611-2614 may transmit theselected motion vector(s) and switch masks to picture/slice switch masksymbol run and/or proxy VLC encoder module 2620. Picture/slice switchmask symbol run and/or proxy VLC encoder module 2620 may compress thereceived data (e.g., motion vector(s) and switch masks) for evaluationvia cost estimator and minimum selector module 2130. Cost estimator andminimum selector module 2130 may determine the best (e.g., lowest bitcount motion vector(s) and/or associated bit masks or the like) andprovide the selection to adaptive motion vector prediction module 1902via best indx. As shown, the selection may also be provided to othermodules as needed for further processing or use by other modules intheir characterizations of various video data. As shown, adaptive motionvector prediction module 1902 may generate a motion vector based on theselected technique, which may be differenced with the original motionvector via differencer 563 to generate motion vector differential (mvd).The motion vector differential may be encoded via adaptive VLC encoderfor motion vector differences module 562 as is discussed further herein.

Further, the generated bit masks may be evaluated and encoded asdiscussed herein. For example, as discussed, compressed bit masks may becompared via cost estimator and minimum selector module 2130 todetermine the best motion vector prediction technique as represented by“best indx”, which may be transmitted to adaptive motion vectorprediction module 1902. The following table illustrates example headers(e.g., overhead) for coding the bit masks using various techniques asdiscussed further herein.

TABLE 2 Overhead for Coding of the Switching Bit Mask Method HeaderHeader Size Bitmap 0 1 Proxy VLC Table 8 10 2 Proxy VLC Table 4 110 3Proxy VLC Table 5 1110 4 Symbol-run Table 8 1111 4

As discussed, symbol-run and/or proxy variable length coding (VLC)methods may include two entropy coders: (1) a symbol-run-based entropycoder and (2) a proxy VLC-based entropy coder. In some examples, asymbol-run coder may compress a bitstream by encoding runs of 0s. Insuch techniques, each value of ‘1’ along with the run of preceding ‘0’smay be coded a single codeword. If there is more than a certain numberof consecutive ‘0’s, an escape (ESC) may be coded and after the lastvalue of ‘1’ is coded an end of block codeword (EOB) may be provided. Insome examples, proxy VLC-based may replace a block of bits of fixedlength in the bitstream with a VLC codeword. In some examples, we usefixed lengths of 2 and/or 3 may be used.

As discussed, a symbol-run coder may compress a bitstream by encodingruns of 0s and a proxy VLC-based coder may replaces a block of bits offixed length in the bitstream with a VLC codeword. Tables 3-6 provideexample codes for use in symbol-run-based and/or proxy VLC-based coding.In Tables 3-6, a separate Table designation may be provided for the sakeof clarity of presentation.

TABLE 3 Symbol-Run Table 8 Code Address Codelength 11 1 2 000 2 3 001 33 100 4 3 011 5 3 0101 6 4 01001 7 5 010000 8 6 101 ESC 3 010001 EOB 6

TABLE 4 Proxy VLC Table 8 Bits VLC Codelength 000 00 2 001 010 3 010 0113 011 100 3 100 101 3 101 1110 4 110 110 3 111 1111 4

TABLE 5 Proxy VLC Table 4 Bits VLC Codelength 000 0 1 001 100 3 010 1013 100 110 3 011 11100 5 101 11101 5 110 11110 5 111 11111 5

TABLE 6 Proxy VLC Table 3 Bits VLC Codelength 00 0 1 01 110 3 10 10 2 11111 3

In some examples there may be few motion vectors to be coded. In suchexamples, it may not be efficient to perform the switching basedprediction as discussed above. In such examples, if the number of motionvectors within a frame is below a pre-selected or heuristicallydetermined threshold, a median motion vector prediction technique may beused for the frame (e.g., mode or method 0 as discussed). In suchexamples, the overhead of 1 bit may be omitted. In some examples, thethreshold may be 16 motion vectors. In some examples, the threshold maybe 8, 10, 20, or 25 motion vectors.

In some examples, entropy coding techniques discussed herein may used toencode the differential motion vector data or bit mask data. In someexamples, an adaptive symbol-run variable coding technique may be used.In some examples, an adaptive proxy variable length coding technique maybe used. Such techniques have been described herein with respect toFIGS. 4 and 6 and will not be repeated for the sake of brevity. In someimplementations, various techniques discussed with respect to adaptivesymbol-run variable coding and/or adaptive proxy variable length codingas discussed elsewhere herein may be used with respect to encoder module306, sub-modules, and the various methods and modes discussed herein.

FIGS. 27(A)-27(D) provide a flow diagram illustrating an example process2700, arranged in accordance with at least some implementations of thepresent disclosure. Process 2700 may include one or more operations,functions or actions as illustrated by one or more of operations 2700-.Process 2700 may form at least part of a next generation video codingprocess. By way of non-limiting example, process 2700 may form at leastpart of a next generation video encoding process as undertaken byencoder system 100 of FIG. 1 and/or entropy encoder module 110 asdiscussed herein.

Process 2700 may begin at operation 2701 Read Input, where frame-levelmotion vector data may be read. Process 2700 may continue at operation2702 Number of MVs<16 (e.g., check for simple case), where it may bedetermined whether the number of non-zero motion vectors for the frameis less than a threshold (e.g., a threshold of 16 is illustrated, thethreshold may be designated as T NOSW PRD). If so, process 2700 maycontinue at operation 2711 and a median motion vector predictiontechnique. If not, process 2700 may continue at operation 2703 EstimateGolomb-Based Bit Cost of Prediction Methods, where prediction methods(e.g., 5 in this example, see Table 1) may be applied to the motionvectors (e.g., neighborhood motion vectors for a block of video data)and Golomb code-based bits estimates may be determined for the methods.

Process 2700 may continue at operation 2704 Select Prediction Method,where the estimated bits for all methods (e.g., modes 1-4) may becompared to determine the one with the smallest cost, which may be theselect prediction method. For example, operation 2704 may determine aframe level motion vector prediction method (based on modes 1-4).Process 2700 may continue at operation 2705 Encode Prediction MethodHeader, where a header-type VLC coding may be used to code the index ofthe selected prediction method (please see Table 2). Process 2700 maycontinue at operation 2706 Default (Median) Prediction?, where it may bedetermined whether the selected prediction method is median prediction.If so, process 2700 may continue at operation 2711 and no switching maskmay need to be determined. If not, process 2700 may continue atoperations 2707-2710, which may encode prediction switching bits, asdiscussed herein. For example, process 2700 may continue at operation2707 Estimate Bits of (Five) Symbol Run/Proxy VLC Methods, where bitsmay be estimated for the Symbol Run/Proxy VLC Methods (please see Table2). Process 2700 may continue at operation 2708 Select Method withSmallest Estimated Bit Cost, where the method with the smallest bit costmay be selected. Process 2700 may continue at operation 2709 Encode theSelected Method Header, where the selected index header may be encoded(please see Table 2). Process 2700 may continue at operation 2710 EncodeSwitching Mask Payload where the switching mask payload may be encoded.As shown, operations 2707-2710 may select an entropy coding method andencode the switching mask.

Process 2700 may continue at operation 2711 Set i=0, from any of2702,2706, or 2710 as described. Process 2700 may begin at operationsgrouping 2712, which may encode sign bits mask to determine (and codethe headers of) the best methods for coding sign bits of motion vectors(with x and y coordinates' signs separately coded). For example,operations grouping 2712 may collect sign bits of an x-coordinate of allmotion vectors in the current frame into a binary mask X and collectsign bits of an y-coordinate of all motion vectors in the current frameinto a binary mask Y. If number of bits in X is less than a thresholddefined as T_bmp (in some examples, for P-frames T_bmp=136, and forF-frames T_bmp=152) then code X with bitmap method. The cost may beestimated for coding X with 7 Proxy VLC methods (described in Table 8below for P-frames, and in Table 9 below for F-frames). The estimatedbits for all coding methods may be compared for X and the one with thesmallest cost may be determined. The header for coding method of X(please see Tables 8 and 9 below) may be encoded. In some examples, Xmay be encoded using the selected method. If number of bits in Y is lessthan a threshold defined as T_bmp (in some examples, for P-framesT_bmp=136, and for F-frames T_bmp=152) then Y may be coded with bitmapmethod. The may be estimated for coding Y with 7 Proxy VLC methods(described in Table 8 below for P-frames, and in Table 9 below forF-frames). The estimated bits for all coding methods may be compared forY and the one with the smallest cost may be determined. The header forcoding method of Y (please see Tables 8 and 9 below) may be encoded. Insome examples, Y may be encoded using the selected method. As discussed,process 2700 may thereby encode an x sign bit and a y sign bit of anyindividual motion vector difference into the bitstream with a sign bitencoding of the plurality of motion vector differences of the videoframe, wherein encoding the sign bit encoding comprises a selected signbit encoding from a plurality of sign bit encoding techniques comprisingproxy variable length coding techniques using different coding tables.

Process 2700 may continue at operation 2713 Get Frame-Based MV ClassesVLC Table Based on History of (up to Four) Last Frames (Tf), where thebest frame-based VLC table for differential motion vector classes codingmay be determined. Please note, classes are described below with respectto FIG. 29 and Table 7. Process 2700 may continue at operations grouping2714, where slice motion vector differentials may be determined. Forexample, slice motion vector differentials may be collected for acurrent slice. As discussed, a plurality of motion vector differenceclasses may be described as a motion vector class table and severalmotion vector class tables may be available on a slice of a video frame.For example a first slice may be associated with a first motion vectorclass table and a second slice of the video frame may be associated witha second motion vector class table.

Process 2700 may continue at decision operation 2715 k<8?, where it maybe determined if the number of motion vectors in the current slice isless than a predetermined or heuristically determined thresholddesignated as T_NOSW_VLC (and set to 8 in this example). If so, a framebased VLC table may be used for motion vector classes coding. If not,process 2700 may continue at operation 2716 Compute the Table (Ts) thathas the Smallest Cost for Slice Si, were the table with the smallestcost may be determined. Process 2700 may continue at operation grouping2717, where a Frame-Based or a Slice-Based motion vector Classes VLCTable Selection may be made such that either a slice-based or aframe-based table for the current slice (with 1 bit flag) or aslice-based table for the current slice (with a 0 bit flag) may beselected based on estimated cost. Process 2700 may continue at operation2718 set j=0.

Process 2700 may continue at operation grouping 2719, where slice-levelmotion vector differentials may be encoded and the code motion vectorclass index may be determined and coded. For example, slice-level motionvector differentials may be encoded for x- and y-components separately.As shown, if class index is in the range 7 through 24, a sub class indexmay be determined and coded. Please note, sub-classes are discussedfurther herein with respect to FIG. 30. In some examples, a vectorcomponents index may be encoded. Process 2700 may continue at decisionoperation 2720 i<N?. If so, process 2700 may continue at operationgrouping 2714 as discussed. If not, the last slice has been processed,and process 2700 may terminate at operation 2721 Finish ProcessingCurrent Frame.

As discussed with respect to process 2700 and elsewhere herein, motionvector differences may be determined for a video frame. Each orindividual motion vector differences may be classified into a selectedclass of multiple motion vector difference classes (as is discussedfurther below with respect to FIG. 29). Within the selected class, anumber of motion vector difference may be available and one may beselected based on each or individual motion vector differences. Forexample, an x- and y-value for an individual motion vector differencemay define the class of the individual motion vector difference (e.g.,based on the magnitude of the individual motion vector difference). Asdiscussed, in some examples, the individual motion vector difference maybe further defined by a sub-class (particularly if the selected class islarge). In either case, a motion vector difference within the class orsub-class may be chosen as associated with the individual motion vectordifference. Via such classification and sub-classification, encodinggains may be recognized. For example, a class identifier associated withthe selected class (and a sub-class identifier associated with theselected sub-class, if needed) and a class-based motion vector indicatorassociated with the indicated class-based motion vector may be encodedinto a bitstream as described.

The bitstream may be received by a decoder which may decode thebitstream to determine a class identifier and a class-based motionvector indicator associated with an individual motion compensated block(e.g., for which the motion vector difference is being determined),determine a decoded motion vector difference for the individual motioncompensated block based on a selected class of multiple motion vectordifference classes such that the selected class is indicated via theclass identifier (and a sub-class may be indicated based on a sub-classidentifier, if needed) and based on the class-based motion vectorindicator. For example, the class-based motion vector indicator mayindicate the decoded motion vector difference within the selected class.As discussed elsewhere herein, a predicted motion vector associated withthe individual motion compensated block may be determined. The predictedmotion vector and the decoded motion vector difference may be added todetermine a final decoded motion vector for the individual motioncompensated block.

FIG. 28 is an illustrative diagram of an example adaptive VLC encoderfor motion vector differences module 662, arranged in accordance with atleast some implementations of the present disclosure. As shown, adaptiveVLC encoder for motion vector differences module 662 may be implementedencoder module 406, which may be implemented via adaptive entropyencoder 110. As shown, adaptive VLC encoder for motion vectordifferences module 662 may include a motion vector differentialclassifier and class look up table module 2801, a sign bits passthroughmodule 2802, a sign bits inverter module 2803, a motion vectordifferential sub-classifier and sub-class look up table module 2804, aproxy VLC sign encoder module 2805, a picture/slice mask collector 2311,a picture/slice mask collector 2312, proxy VLC look up tables 2820,and/or a cost estimator and minimum selector module 2630. As discussedabove, in some examples, cost estimator and minimum selector module 2630may be implemented via adaptive motion prediction analyzer module 1901.In various examples, cost estimator and minimum selector module 2630 maybe implemented via one of or both of adaptive VLC encoder for motionvector differences module 662 and adaptive motion prediction analyzermodule 1901.

As shown, motion vector differential data (mvd) may be received viaadaptive VLC encoder for motion vector differences module 662, which mayprocess the motion vector differential data (mvd) to provide compressed(or coded) video data 426, which may be transmitted to bitstreamassembler 408 (please see FIG. 4).

For example, the techniques discussed herein may include classificationof MV differentials (mvd) into one of multiple (in some examples, many)available categories (e.g., classes) via motion vector differentialclassifier and class look up table module 280, for example, forefficient representation and, in some examples, sub-classification viamotion vector differential sub-classifier and sub-class look up tablemodule 2804, for example, into one of several sub-classes. Class andsubclass components may be encoded and sent via bits of compressed videodata 426 to identify the magnitude of the motion vector differential,for example. The sign of motion vector differential may coded separatelyvia sign bits passthrough module 2802, sign bits inverter module 2803,picture/slice mask collectors 2311, 2812 and further compressed byseveral available proxy VLC look up tables 2820 and then analyzed viacost estimator and minimum selector module 2630 for the most compressedrepresentation.

For example, the encoding process of motion vector differentials may beprovided as follows. Transmitted values may be the followingdifferential vectors:

-   MVDx=MVx-PMVx aMVDx=abs(MVDx)-   MVDy=MVy-PMVy aMVDx=abs(MVDx)    where, MVx may be actual (e.g., original) motion vector, and PMVx    may be the predicted motion vector. In some examples, MVDx and MVDy    may have the values from −2047 to 2047.

Further, in some examples, MVDx, MVDy (e.g., the components of motionvector differentials may be coded jointly. In some examples, thepossible differential motion vector pairs (MVDx, MVDy) create a2-dimensional plane and each pair may belong to a class. For example, 34classes may be specified as shown in FIG. 27 and Table 7.

FIG. 29 is an illustrative diagram of an example differential motionvector classes 2900, arranged in accordance with at least someimplementations of the present disclosure. As shown in FIG. 29,differential motion vector classes 2900 may be established withcharacteristic ranges as provided in Table 7.

TABLE 7 Motion Vector Differential Ranges for MVD Classes MVD min classaMVDx max aMVDx min aMVDy max aMVDy Size 0 0 0 0 0 1 × 1 1 1 1 0 0 1 × 12 0 0 1 1 1 × 1 3 1 1 1 1 1 × 1 4 2 3 0 1 2 × 2 5 0 1 2 3 2 × 2 6 2 3 23 2 × 2 7 4 7 0 3 4 × 4 8 8 15 0 7 8 × 8 9 16 31 0 15 16 × 16 10 32 63 031 32 × 32 11 64 127 0 63 64 × 64 12 128 255 0 127 128 × 128 13 256 5110 255 256 × 256 14 512 1023 0 511 512 × 512 15 1024 2047 0 1023 1024 ×1024 16 0 3 4 7 4 × 4 17 0 7 8 15 8 × 8 18 0 15 16 31 16 × 16 19 0 31 3263 32 × 32 20 0 63 64 127 64 × 64 21 0 127 128 255 128 × 128 22 0 255256 511 256 × 256 23 0 511 512 1023 512 × 512 24 0 1023 1024 2047 1024 ×1024 25 4 7 4 7 4 × 4 26 8 15 8 15 8 × 8 27 16 31 16 31 16 × 16 28 32 6332 63 32 × 32 29 64 127 64 127 64 × 64 30 128 255 128 255 128 × 128 31256 511 256 511 256 × 256 32 512 1023 512 1023 512 × 512 33 1024 20471024 2047 1024 × 1024

In some examples, the motion vector differentials may be furthercharacterized via sub-classes. For example, classes 7-15 may have ahorizontal shape type sub-class such that each sub-class may divide theclass as shown in FIG. 30(A) and classes 16-24 may have a vertical shapetype sub-class such that each sub-class may divide the class as shown inFIG. 30(B).

FIGS. 30(A) and 30(B) are illustrative diagrams of example differentialmotion vector subclasses 3001 and 3002, arranged in accordance with atleast some implementations of the present disclosure. As shown in FIG.30(A), vertical sub-classes 3001 may divide a class into horizontal typesub-classes labeled sub-classes 0-1. As shown in FIG. 30(A), verticalsub-classes 3001 may divide a class into horizontal type sub-classeslabeled sub-classes 0-1.

For example, as shown, classes 4-6 may have a size of 2×2 (4 motionvectors). Such classes may be divided into 4 sub-classes such that eachsub-class may be given an appropriate codeword. For example, thecodewords for sub-classes classes 4-6 may be selected based oncorresponding statistics from previous frame(s). In some examples, ahistory of up to 4 frames (e.g., a history of 1-4 frames) may be used.Other classes (related by size or otherwise) may be provided codewordsin a similar manner.

Each MVD pair as discussed herein may may include a VLC main class, aVLC sub-class, sub-class component bit(s), and sign bits for the MVDpair. VLC tables for main classes are illustrated next. In someexamples, there may be 10 tables for P and 10 tables for F frames(indexed 0 through 9). For example, the table may be selected in anadaptive way such that each slice may use a different table. At eachslice, for example two table indices may be determined: a frame-basedtable index and slice-based table index, as discussed above with respectto FIG. 27. The frame-based table index at first frame may be set to 0.At following frames, the frame-based table index may be set to an indexof the table that yields the smallest bit count computed from thegathered statistics of previous frames of the same type. In someexamples, the last 4 pictures' statistics may be used. Further to thistable index, at each slice, a slice-based table may be determined as thebest table (out of the best 8 frame-based tables in the illustrativeexample) applied to the motion vectors of the current slice. If thedetermined table plus the overhead needed to indicate it produce fewerbits than the frame-based table, for example, class information may becoded with the slice-based table within that slice; otherwise, aframe-based table may be used. The selection between frame-based tableand slice-based table at each slice may be coded with a 1 bit overheadin some examples. Further, in some examples, 2×2 size sub-classes may becoded either with 2 bit codes or with 1-3 bit codes depending on thehistory (e.g., up to four last frames). However, in some examples,horizontal and vertical sub-classes may always be coded with thefollowing VLC codes:

Sub-class 0: ‘0’

Sub-class 1: ‘10’

Sub-class 2: ‘110’

Sub-class 3: ‘111’

In some examples, to specify a motion vector differential pair in aclass (or sub-class), the number of required bits may be given as:N=Log2(w×h),where w=width of a class (sub-class), h=height of a class (sub-class).

At the end of a slice, sign bits of the vectors in that slice may becoded. In some examples, the sign bits may be accumulated as follows. Ifthe values of MVDx or MVDy are non-zero, sign bits are appended to asign bits mask. In some examples, two sign bits masks may be created,one per x coordinate and one per y coordinate. Such separating of themasks coordinate-wise may provide increased correlation within eachmask. For example, the sign bit masks may be populated as follows:

If MVD(x)=0, no sign bit added to sign bits mask x

If MVD(x)>0, ‘0’ sign bit added to sign bits mask x

If MVD(x)<0, ‘1’ sign bit added to sign bits mask x

If MVD(y)=0, no sign bit added to sign bits mask y

If MVD(y)>0, ‘0’ sign bit added to sign bits mask y

If MVD(y)<0, ‘1’ sign bit added to sign bits mask y

Once all sign bits for the current slice are accumulated, the sign bitsmasks may be coded using either bitmap coding or a proxy VLC approach,as described herein. For example, the following table provides overheadfor coding of the sign bits masks (both x and y) in P-frames andF-frames.

TABLE 8 Overhead for coding of the sign bits masks (both x and y) inP-frames Method Header Header Size Bitmap 0 1 Proxy VLC Table 0 1100 4Proxy VLC Table 1 1101 4 Proxy VLC Table 2 1110 4 Inverted Proxy VLCTable 0 100 3 Inverted Proxy VLC Table 1 101 3 Inverted Proxy VLC Table2 1111 4

TABLE 9 Overhead for coding of the sign bits masks (both x and y) inF-frames Method Header Header Size Bitmap 0 1 Proxy VLC Table 0 1100 4Proxy VLC Table 1 1101 4 Proxy VLC Table 2 1110 4 Inverted Proxy VLCTable 0 100 3 Inverted Proxy VLC Table 1 101 3 Inverted Proxy VLC Table2 1111 4

As discussed above, if the number of motion vectors in the current frameis less than a threshold (T_bmp), the masks may be automatically codedas bitmap without any overhead included. As discussed, in someimplementations, the T_bmp thresholds for P- and F-frames may be set to136 and 152, respectively. Otherwise, overhead tables Table 8 and Table9 may be used to indicate a coding method for masks x and y. Forexample, the VLC tables used for proxy VLC approach of coding the signbits masks are shown in Tables 10-12.

TABLE 10 Proxy VLC Table 0 Bits VLC Codelength 000 0 1 001 100 3 0101010 4 100 1011 4 011 1100 4 101 1101 4 110 1110 4 111 1111 4

TABLE 11 Proxy VLC Table 1 Bits VLC Codelength 000 00 2 001 010 3 010011 3 100 100 3 011 1110 4 101 101 3 110 110 3 111 1111 4

TABLE 12 Proxy VLC Table 2 Bits VLC Codelength 000 00 2 001 010 3 010011 3 100 100 3 011 101 4 101 110 3 110 1110 3 111 1111 4

In some examples, once a slice is coded, the next slice may beprocessed. If the next slice is in the same frame, it may not benecessary to re-compute the frame-based best table index in someexamples. If the current frame is finished processing and the next sliceis part of the following frame, the statistics may be updated and aframe based index may be determined as discussed above.

As discussed above, if there are few motion vectors to be coded, it maynot be efficient to perform the switching based coding as justdescribed. For example, if the number of motion vectors within a frameis below a pre-selected threshold, an initial coding method may be usedfor prediction. In such examples, the method may use a frame-based VLCtable selection without paying any overhead. In some examples, theselected threshold may be T NOSW VLC=8 such that the Initial CodingMethod is used for prediction if less than 8 MVs are coded per frame.

Using the system(s) and techniques discussed herein, efficient tradeoffsbetween gains achievable and complexity may be achieved.

As discussed with respect to FIG. 16, example video coding system 1600may include imaging device(s) 1601, video encoder 100 and/or a videoencoder implemented via logic circuitry 1650 of processing unit(s) 1620,video decoder 200 and/or a video decoder implemented via logic circuitry1650 of processing unit(s) 1620, an antenna 1602, one or moreprocessor(s) 1603, one or more memory store(s) 2004, and/or a displaydevice 2005.

In some examples, video encoder 100 implemented via logic circuitry 1650may include an image buffer (e.g., via either processing unit(s) 1620 ormemory store(s) 1604)) and a graphics processing unit (e.g., viaprocessing unit(s) 1620). The graphics processing unit may becommunicatively coupled to the image buffer. The graphics processingunit may include video encoder 100 as implemented via logic circuitry1650 to embody the various modules as discussed herein. For example, thegraphics processing unit may include entropy encoder logic circuitry,and so on. The logic circuitry may be configured to perform the variousoperations as discussed herein. For example, the entropy encoder logiccircuitry may be configured to determine a selected motion vectorprediction technique for a block of video data from a plurality ofmotion vector prediction techniques, generate a prediction motion vectorfor the block of video data based at least in part on the selectedmotion vector prediction technique, difference the prediction motionvector with an original motion vector associated with the block of videodata to generate a motion vector differential, and entropy encode themotion vector differential to generate coded motion vector data. Videodecoder 200 may be implemented in a similar manner.

In some examples, antenna 1602 of video coding system 1600 may beconfigured to receive an entropy encoded bitstream of video data. Asdiscussed, the bitstream may include compressed video data of varioustypes. Video coding system 1600 may also include video decoder 200coupled to antenna 1602 and configured to decode the encoded bitstream.For example, video decoder 200 may be configured to disassemble theentropy encoded bitstream to determine coded video data, decode thecoded video data to determine a difference motion vector associated witha block of video data, determine a selected motion vector predictiontechnique for the block of video data from a plurality of motion vectorprediction techniques, generate a prediction motion vector for the blockof video data based at least in part on the selected motion vectorprediction technique, add the difference motion vector and the predictedmotion vector to generate a reconstructed motion vector for the block ofvideo data, and generate a video frame based at least in part on thereconstructed motion vector. The video frame may be configured to bepresented to a user via a display device, for example.

As discussed, video data of different types may be entropy coded togenerate compressed (e.g., coded) video data. The compressed video datamay be assembled to generate an output bitstream. With reference to FIG.8, example bitstream 800 may include indicator data 801, compressedvideo data 321, compressed video data 322, compressed video data 323,compressed video data 324, compressed video data 325, compressed videodata 326, and/or compressed video data 327. For example, compressedvideo data 326 may include coded motion vector data, coded motion vectordifferentials, indicator data indicating motion vector prediction, a bitmask, indicator data indicating an entropy coding technique, motionvector differential classification or sub-classification data,x-differential data, y-differential data, x-absolute value differentialdata, y-absolute value differential data, jointly coded motion vectordifferential pair data, class component bits, sub-class component bits,or sign bits, or the like, as discussed herein.

While certain features set forth herein have been described withreference to various implementations, this description is not intendedto be construed in a limiting sense. Hence, various modifications of theimplementations described herein, as well as other implementations,which are apparent to persons skilled in the art to which the presentdisclosure pertains are deemed to lie within the spirit and scope of thepresent disclosure.

The following examples pertain to further embodiments.

In an example, a computer-implemented method for video coding mayinclude receiving motion vector data including a plurality of motionvectors for a video frame, determining a first predicted motion vectorusing a median motion vector prediction technique for each motioncompensated block of the video frame, determining a second predictedmotion vector using a selected motion vector prediction technique foreach motion compensated block, selecting the first or the secondpredicted motion vector for each motion compensated block and defining amotion vector bit mask identifying the median or the selected motionvector prediction technique for each motion compensated block,determining a selected coding method for the motion vector bit mask, andencoding the motion vector bit mask into a bitstream based on theselected coding method.

In another example, a computer-implemented method for video coding mayfurther include determining a frame level motion vector predictionmethod based on a bit cost analysis of a plurality of frame level motionvector prediction methods, wherein the frame level motion vectorprediction method defines the selected motion vector predictiontechnique, differencing the plurality of motion vectors with anassociated selected predicted motion vector for each motion compensatedblock to generate a plurality of motion vector differences, and coding aframe level motion vector prediction method header, a selected codingmethod header, and an encoded payload comprising the plurality of motionvector differences into a bitstream. The selected motion vectorprediction comprises at least one of a Graham motion vector predictiontechnique, a Graham last block motion vector prediction technique, or aspiral motion vector prediction technique. The selected motion vectorprediction technique may include a spiral motion vector predictiontechnique when the first predicted motion vector is a zero motion vectorand a Graham motion vector prediction technique when the first predictedmotion vector is a non-zero motion vector. The selected motion vectorprediction technique may include a spiral motion vector predictiontechnique when the first predicted motion vector is a zero motion vectorand a Graham last block motion vector prediction technique when thefirst predicted motion vector is a non-zero motion vector. The selectedmotion vector prediction technique may include a Graham motion vectorprediction technique, and wherein, if the difference between first andsecond prediction motion vectors for a first motion compensated block iswithin a threshold, the selected motion vector for the first motioncompensated block is set to the first prediction motion vector. Theselected motion vector prediction technique may include a Graham motionvector prediction technique, and wherein, if the difference betweenfirst and second prediction motion vectors for a first motioncompensated block are within a threshold, the selected motion vector forthe first motion compensated block is set to the second predictionmotion vector. The selected motion vector prediction technique mayinclude a Graham last block motion vector prediction technique, andwherein, if the difference between first and second prediction motionvectors for a first motion compensated block is within a threshold, theselected motion vector for the first motion compensated block is set tothe first prediction motion vector. The selected motion vectorprediction technique may include a Graham last block motion vectorprediction technique, and wherein, if the difference between first andsecond prediction motion vectors for a first motion compensated blockare within a threshold, the selected motion vector for the first motioncompensated block is set to the second prediction motion vector. Theselected coding method may include at least one of an adaptivesymbol-run variable length coding technique or an adaptive proxyvariable length coding technique. The selected motion vector predictionmay include a spiral motion vector prediction technique, and wherein thespiral motion vector prediction technique comprises scanning theneighboring original motion vectors associated with neighboring blocksof video data in a spiral pattern around the block of video data todetermine a first non-zero neighboring original motion vector andgenerating the prediction motion vector for the block of video data asthe first non-zero neighboring original motion vector.

In an example, a computer-implemented method for video coding mayinclude determining a plurality of motion vector differences for a videoframe, classifying an individual motion vector difference of theplurality of motion vector differences into a selected class of aplurality of motion vector difference classes, indicating a class-basedmotion vector within the selected class associated with the individualmotion vector difference, and encoding a class identifier associatedwith the selected class and a class-based motion vector indicatorassociated with the indicated class-based motion vector into abitstream.

In another example, a computer-implemented method for video coding mayfurther include determining whether the selected class comprisessub-classes, sub-classifying, when the selected class comprisessub-classes, the individual motion vector difference into a selectedsub-class of a plurality of motion vector difference sub-classes withinthe selected class, wherein the class-based motion vector is within theselected sub-class, encoding a sub-class identifier associated with theselected sub-class into the bitstream, and encoding an x sign bit and ay sign bit of the individual motion vector difference into the bitstreamwith a sign bit encoding of the plurality of motion vector differencesof the video frame, wherein encoding the sign bit encoding comprises aselected sign bit encoding from a plurality of sign bit encodingtechniques comprising proxy variable length coding techniques usingdifferent coding tables. The plurality of motion vector differenceclasses may include a motion vector class table and wherein the motionvector class table is selected from a plurality of motion vector classtables, wherein the motion vector class table is associated with a sliceof the video frame, and wherein a second slice of the video frame isassociated with a second motion vector class table of the plurality ofmotion vector class tables.

In an example, a computer-implemented method for video coding mayinclude determining a selected motion vector prediction technique for ablock of video data from a plurality of motion vector predictiontechniques, generating a prediction motion vector for the block of videodata based at least in part on the selected motion vector predictiontechnique, differencing the prediction motion vector with an originalmotion vector associated with the block of video data to generate amotion vector differential, and entropy encoding the motion vectordifferential to generate coded motion vector data.

In another example, a computer-implemented method for video coding mayfurther include receiving video data for encoding such that the videodata may include the block of video data, neighboring original motionvectors associated with neighboring blocks of video data such that theneighboring blocks of video data neighbor the block of video data, andthe original motion vector associated with the block of video data,classifying the motion vector differential into an entropy coding classand an entropy coding sub-class such that the motion vector differentialmay include at least one of an x-differential, a y-differential, anx-absolute value differential, or a y-absolute value differential,entropy encoding a switching bit mask via at least one of a bitmapencoding technique, a proxy variable length coding technique, or asymbol-run coding technique, and assembling the coded motion vector datawith compressed video data to generate an output bitstream. Entropyencoding the motion vector differential may include entropy encoding themotion vector differential jointly as a motion vector differential pairand wherein the coded motion vector data comprise a variable length codeof the entropy coding class, a variable length code of the entropycoding sub-class, sub-class component bits, and sign bits. Entropyencoding the motion vector differential may include at least one of anadaptive symbol-run variable length coding technique or an adaptiveproxy variable length coding technique, such that the adaptivesymbol-run variable length coding technique comprises encoding a run ofskipped blocks, and such that the adaptive proxy variable length codingtechnique may include replacing original fixed length 1-dimensionalblocks with variable length codes, such that the original fixed length1-dimensional blocks may include a length of at least one of 2 bits, 3bits, or 4 bits, such that the original fixed length 1-dimensionalblocks may include a length of 2 bits and the variable length codes mayinclude a variable length of 1 to 3 bits, and such that the originalfixed length 1-dimensional blocks may include a length of 3 bits and thevariable length codes may include a variable length of 1 to 5 bits. Theplurality of motion vector prediction techniques may include at leastone of a median motion vector prediction technique, a Graham motionvector prediction technique, a Graham last block motion vectorprediction technique, or a spiral motion vector prediction technique.The median motion vector prediction technique may include determining afirst neighboring original motion vector for a first neighboring blockof video data above and aligned with a left edge of the block of videodata, a second neighboring original motion vector for a secondneighboring block of video data to the left of and above the block ofvideo data, and a third neighboring original motion vector for a thirdneighboring block of video data to the left of and aligned with a topedge of the block of video data, and generating the prediction motionvector for the block of video data as the median of the first, secondand third neighboring original motion vectors. The Graham motion vectorprediction technique may include determining a first neighboringoriginal motion vector for a first neighboring block of video data aboveand aligned with a left edge of the block of video data, a secondneighboring original motion vector for a second neighboring block ofvideo data to the left of and above the block of video data, and a thirdneighboring original motion vector for a third neighboring block ofvideo data to the left of and aligned with a top edge of the block ofvideo data, determining an absolute value difference between the secondand the third neighboring original motion vectors, determining anabsolute value difference between the first and the second neighboringoriginal motion vectors, and generating the prediction motion vector forthe block of video data as the first neighboring original motion vectorif the absolute value difference between the second and the thirdneighboring original motion vectors is less than or equal to theabsolute value difference between the first and the second neighboringoriginal motion vectors and as the third neighboring original motionvector if the absolute value difference between the second and the thirdneighboring original motion vectors is greater than the absolute valuedifference between the first and the second neighboring original motionvectors. The Graham last block motion vector prediction technique mayinclude determining a first neighboring original motion vector for afirst neighboring block of video data above and aligned with a rightedge of the block of video data, a second neighboring original motionvector for a second neighboring block of video data to the left of andabove the block of video data, and a third neighboring original motionvector for a third neighboring block of video data to the left of andaligned with a bottom edge of the block of video data, determining anabsolute value difference between the second and the third neighboringoriginal motion vectors, determining an absolute value differencebetween the first and the second neighboring original motion vectors,and generating the prediction motion vector for the block of video dataas the first neighboring original motion vector if the absolute valuedifference between the second and the third neighboring original motionvectors is less than or equal to the absolute value difference betweenthe first and the second neighboring original motion vectors and as thethird neighboring original motion vector if the absolute valuedifference between the second and the third neighboring original motionvectors is greater than the absolute value difference between the firstand the second neighboring original motion vectors. The spiral motionvector prediction technique may include scanning the neighboringoriginal motion vectors associated with neighboring blocks of video datain a spiral pattern around the block of video data to determine a firstnon-zero neighboring original motion vector and generating theprediction motion vector for the block of video data as the firstnon-zero neighboring original motion vector. Determining the selectedmotion vector prediction technique for a block of video data from aplurality of motion vector prediction techniques may include determininga first prediction motion vector via a first prediction motion vectorprediction technique such that the first prediction motion vectorprediction technique may include the median motion vector predictiontechnique, when the first prediction motion vector is a zero vectordetermining a second prediction motion vector via a second predictionmotion vector prediction technique such that the second predictionmotion vector prediction technique comprises the spiral motion vectorprediction technique, when the first prediction motion vector is anon-zero vector determining a second prediction motion vector via athird prediction motion vector prediction technique such that the thirdprediction motion vector prediction technique may include at least oneof the Graham motion vector prediction technique or the Graham lastblock motion vector prediction technique, when a difference between thefirst prediction motion vector and the second prediction motion vectoris less than a threshold, in a first mode, determining the selectedmotion vector prediction technique as the first motion vector predictiontechnique, and, in a second mode, determining the selected motion vectorprediction technique as at least one of the second or third motionvector prediction techniques, and when a difference between the firstprediction motion vector and the second prediction motion vector isgreater than a threshold determining a first estimated bit costassociated with the first motion vector prediction technique,determining a second estimated bit cost associated with the second orthird motion vector prediction technique, and, when the first estimatedbit cost is less than the second estimated bit cost, determining theselected motion vector prediction technique as the first motion vectorprediction technique, and, when the second estimated bit cost is lessthan the first estimated bit cost, determining the selected motionvector prediction technique as the second or third motion vectorprediction technique, and adding a switching bit to the switching bitmask

In an example, a computer-implemented method for determining aprediction motion vector for a current block of video data may includedetermining a pattern to provide a sequence of neighboring blocks of ablock of video data such that the pattern may include a substantiallyspiral shape, scanning original motion vectors associated with theneighboring blocks according to the sequence for a first non-zerooriginal motion vector, and, when the first non-zero original motionvector is determined, generating a prediction motion vector associatedwith the block of video data as the first non-zero original motionvector, and, when no non-zero original is determined, generating theprediction motion vector associated with the block of video data as azero vector. In further example methods, the pattern may include a shapebeginning at a first neighboring block to the left of and aligned withthe top of the block of video data and progressing upwards to a secondblock to the left of and above the block of video data, then to theright along the top of the block of video data to a third neighboringblock to the right of and above the block of video data, then downwardsto a fourth neighboring block to the right of and below the block ofvideo data, then left to a fifth neighboring block to the left of andbelow the block of video data, the pattern may include a shape beginningat a first neighboring block to the left of and aligned with the top ofthe block of video data and progressing downwards to a secondneighboring block to the left of and aligned with the bottom of theblock of video data, then skipping to a third neighboring block aboveand aligned with the left of the block of video data, then progressingright to a fourth neighboring block above and aligned with the rightside of the block of video data, then skipping to a fifth neighboringblock to the left of above the block of video data, then skipping to asixth neighboring block to the right of and above the block of videodata, then skipping to a seventh neighboring block to the left of belowthe block of video data, then skipping to an eighth neighboring block tothe right of and aligned with the top of the block of video data, thenprogressing downward to a ninth neighboring block to the right of andaligned with the bottom of the block of video data, the neighboringblocks may include a size of 4×4, or the neighboring blocks may include20 neighboring blocks.

In other examples, a video encoder may include an image buffer and agraphics processing unit having entropy encoder logic circuitry. Thegraphics processing unit may be communicatively coupled to the imagebuffer and the entropy encoder logic circuitry may be configured toreceive motion vector data comprising a plurality of motion vectors fora video frame, determine a first predicted motion vector using a medianmotion vector prediction technique for each motion compensated block ofthe video frame, determine a second predicted motion vector using aselected motion vector prediction technique for each motion compensatedblock, select the first or the second predicted motion vector for eachmotion compensated block and defining a motion vector bit maskidentifying the median or the selected motion vector predictiontechnique for each motion compensated block, determine a selected codingmethod for the motion vector bit mask, and encode the motion vector bitmask into a bitstream based on the selected coding method.

In a further example video encoder, the entropy encoder logic circuitrymay be further configured to determine a frame level motion vectorprediction method based on a bit cost analysis of a plurality of framelevel motion vector prediction methods, wherein the frame level motionvector prediction method defines the selected motion vector predictiontechnique, difference the plurality of motion vectors with an associatedselected predicted motion vector for each motion compensated block togenerate a plurality of motion vector differences, and code a framelevel motion vector prediction method header, a selected coding methodheader, and an encoded payload comprising the plurality of motion vectordifferences into a bitstream. The selected motion vector predictioncomprises at least one of a Graham motion vector prediction technique, aGraham last block motion vector prediction technique, or a spiral motionvector prediction technique. The selected motion vector predictiontechnique may include a spiral motion vector prediction technique whenthe first predicted motion vector is a zero motion vector and a Grahammotion vector prediction technique when the first predicted motionvector is a non-zero motion vector. The selected motion vectorprediction technique may include a spiral motion vector predictiontechnique when the first predicted motion vector is a zero motion vectorand a Graham last block motion vector prediction technique when thefirst predicted motion vector is a non-zero motion vector. The selectedmotion vector prediction technique may include a Graham motion vectorprediction technique, and wherein, if the difference between first andsecond prediction motion vectors for a first motion compensated block iswithin a threshold, the selected motion vector for the first motioncompensated block is set to the first prediction motion vector. Theselected motion vector prediction technique may include a Graham motionvector prediction technique, and wherein, if the difference betweenfirst and second prediction motion vectors for a first motioncompensated block are within a threshold, the selected motion vector forthe first motion compensated block is set to the second predictionmotion vector. The selected motion vector prediction technique mayinclude a Graham last block motion vector prediction technique, andwherein, if the difference between first and second prediction motionvectors for a first motion compensated block is within a threshold, theselected motion vector for the first motion compensated block is set tothe first prediction motion vector. The selected motion vectorprediction technique may include a Graham last block motion vectorprediction technique, and wherein, if the difference between first andsecond prediction motion vectors for a first motion compensated blockare within a threshold, the selected motion vector for the first motioncompensated block is set to the second prediction motion vector. Theselected coding method may include at least one of an adaptivesymbol-run variable length coding technique or an adaptive proxyvariable length coding technique. The selected motion vector predictionmay include a spiral motion vector prediction technique, and wherein thespiral motion vector prediction technique comprises scanning theneighboring original motion vectors associated with neighboring blocksof video data in a spiral pattern around the block of video data todetermine a first non-zero neighboring original motion vector andgenerating the prediction motion vector for the block of video data asthe first non-zero neighboring original motion vector.

In other examples, a video encoder may include an image buffer and agraphics processing unit having entropy encoder logic circuitry. Thegraphics processing unit may be communicatively coupled to the imagebuffer and the entropy encoder logic circuitry may be configured todetermine a plurality of motion vector differences for a video frame,classify an individual motion vector difference of the plurality ofmotion vector differences into a selected class of a plurality of motionvector difference classes, indicate a class-based motion vector withinthe selected class associated with the individual motion vectordifference, and encode a class identifier associated with the selectedclass and a class-based motion vector indicator associated with theindicated class-based motion vector into a bitstream.

In a further example video encoder, the entropy encoder logic circuitrymay be further configured to determine whether selected class comprisessub-classes, sub-classify, when the selected class comprisessub-classes, the individual motion vector difference into a selectedsub-class of a plurality of motion vector difference sub-classes withinthe selected class, wherein the class-based motion vector is within theselected sub-class, encode a sub-class identifier associated with theselected sub-class into the bitstream, and encode an x sign bit and a ysign bit of the individual motion vector difference into the bitstreamwith a sign bit encoding of the plurality of motion vector differencesof the video frame, wherein encoding the sign bit encoding comprises aselected sign bit encoding from a plurality of sign bit encodingtechniques comprising proxy variable length coding techniques usingdifferent coding tables. The plurality of motion vector differenceclasses may include a motion vector class table and wherein the motionvector class table is selected from a plurality of motion vector classtables, wherein the motion vector class table is associated with a sliceof the video frame, and wherein a second slice of the video frame isassociated with a second motion vector class table of the plurality ofmotion vector class tables.

In yet another example, a system may include a video decodercommunicatively coupled to the antenna and configured to decode anentropy encoded bitstream to determine a frame level motion vectorprediction method and a motion vector bit mask for a video frame,wherein the motion vector bit mask identifies a median or a selectedmotion vector prediction technique for each motion compensated block ofthe video frame, and wherein the selected motion vector predictiontechnique is based on the frame level motion vector prediction methodand a first motion vector for each motion compensated block determinedbased on the median motion vector prediction technique, determine basedon the motion vector bit mask the individual motion compensated block isassociated with the median motion vector prediction technique and thesecond individual motion compensated block is associated with theselected motion vector prediction technique, generate a first predictedmotion vector associated with the individual motion compensated blockbased on the median motion vector prediction technique, generate amedian based prediction motion vector associated with the secondindividual motion compensated block, and generate a second predictedmotion vector associated with the second motion compensated block basedon a spiral motion vector prediction technique when the median basedprediction motion vector is a zero motion vector and based on at leastone of a Graham motion vector prediction technique or a Graham lastblock motion vector prediction technique when the median basedprediction motion vector is a non-zero motion vector.

In a further example system, the system may include an antennaconfigured to receive the entropy encoded bitstream of video data and adisplay device configured to present a video frame. The video decodermay be further configured to decode the entropy encoded bitstream todetermine a first difference motion vector associated with the firstindividual motion compensated block and a second difference motionvector associated with the second individual motion compensated block ofthe video frame, add the first difference motion vector and the firstpredicted motion vector to determine a first final motion vector for thefirst individual motion compensated block, and add the second differencemotion vector and the second predicted motion vector to determine afirst final motion vector for the first individual motion compensatedblock.

In yet another example, a system may include a video decodercommunicatively coupled to the antenna and configured to decode anentropy encoded bitstream to decode an entropy encoded bitstream todetermine a class identifier and a class-based motion vector indicatorassociated with an individual motion compensated block, determine adecoded motion vector difference for the individual motion compensatedblock based on a selected class of a plurality of motion vectordifference classes, wherein the selected class is indicated via theclass identifier, and based on the class-based motion vector indicator,wherein the class-based motion vector indicator indicates the decodedmotion vector difference within the selected class, determine apredicted motion vector associated with the individual motioncompensated block, and add the predicted motion vector and the decodedmotion vector difference to determine a final decoded motion vector forthe individual motion compensated block.

In a further example system, the system may include an antennaconfigured to receive the entropy encoded bitstream of video data and adisplay device configured to present a video frame. The video decodermay be further configured to decode the entropy encoded bitstream todetermine a sub-class identifier associated with the individual motioncompensated block, wherein the selected class comprises sub-classes, andwherein to determine the decoded motion vector difference for theindividual motion compensated block comprises the video decoder beingconfigured to determine the decoded motion vector difference from asub-class identified by the sub-class identifier.

In a further example, at least one machine readable medium may include aplurality of instructions that in response to being executed on acomputing device, causes the computing device to perform the methodaccording to any one of the above examples.

In a still further example, an apparatus may include means forperforming the methods according to any one of the above examples.

The above examples may include specific combination of features.However, such the above examples are not limited in this regard and, invarious implementations, the above examples may include the undertakingonly a subset of such features, undertaking a different order of suchfeatures, undertaking a different combination of such features, and/orundertaking additional features than those features explicitly listed.For example, all features described with respect to the example methodsmay be implemented with respect to the example apparatus, the examplesystems, and/or the example articles, and vice versa.

What is claimed:
 1. A video encoder comprising: a memory to store videodata; and a processor coupled to the memory, the processor to: determinea pattern to provide a sequence of neighboring blocks of a block of thevideo data, wherein the pattern comprises a substantially spiral shape;scan original motion vectors associated with the neighboring blocksaccording to the sequence for a first non-zero original motion vector;and when the first non-zero original motion vector is determined, togenerate a prediction motion vector associated with the block as thefirst non-zero original motion vector; or when no non-zero original isdetermined, to generate the prediction motion vector associated with theblock as a zero motion vector; and to generate an output bitstream basedin part on the prediction motion vector for the block.
 2. The videoencoder of claim 1, wherein the pattern comprises a shape beginning at afirst neighboring block of the block and progressing through neighboringblocks clockwise around the block.
 3. The video encoder of claim 1,wherein the pattern comprises a shape beginning at a first neighboringblock to the left of and aligned with the top of the block andprogressing through neighboring blocks clockwise around the block. 4.The video encoder of claim 1, wherein the pattern comprises a shapebeginning at a first neighboring block to the left of and aligned withthe top of the block and progressing upwards to a second block to theleft of and above the block, then to the right along the top of theblock to a third neighboring block to the right of and above the block,then downwards to a fourth neighboring block to the right of and belowthe block, then left to a fifth neighboring block to the left of andbelow the block.
 5. The video encoder of claim 1, wherein the patterncomprises a shape beginning at a first immediately adjacent neighboringof the block, progressing clockwise through all remaining firstimmediately adjacent neighboring blocks of the block, and thenprogressing clockwise through second neighboring blocks immediatelyadjacent to first immediately adjacent neighboring blocks.
 6. The videoencoder of claim 1, wherein the pattern comprises a shape beginning at afirst neighboring block to the left of and aligned with the top of theblock and progressing downwards to a second neighboring block to theleft of and aligned with the bottom of the block, then skipping to athird neighboring block above and aligned with the left of the block,then progressing right to a fourth neighboring block above and alignedwith the right side of the block, then skipping to a fifth neighboringblock to the left of above the block, then skipping to a sixthneighboring block to the right of and above the block, then skipping toa seventh neighboring block to the left of below the block, thenskipping to an eighth neighboring block to the right of and below theblock, then skipping to a ninth neighboring block to the right of andaligned with the top of the block, then progressing downward to a tenthneighboring block to the right of and aligned with the bottom of theblock.
 7. The video encoder of claim 1, wherein the processor is furtherto: determine a second prediction motion vector for a second block ofthe video data using a median motion vector prediction technique basedon second motion vectors associated with second neighboring blocks ofthe second block; and when the second prediction motion vector is a zeromotion vector, scan third motion vectors associated with thirdneighboring blocks of the second block according to the sequence togenerate a third prediction motion vector for the second block; or whenthe second prediction motion vector is a non-zero motion vector,determine a third prediction motion vector for the second block usingone of a Graham motion vector prediction technique or a Graham lastblock motion vector prediction technique.
 8. The video encoder of claim7, wherein the processor is further to: compare a difference between thesecond prediction motion vector and the third prediction motion vectorto a threshold; and when the difference is less than the threshold,select the second prediction motion vector for the second block; or whenthe difference is not less than the threshold, estimate a first bit costcorresponding to the second prediction motion vector and a second bitcost corresponding to the third prediction motion vector and select thesecond or third prediction motion vector for the second block based onthe first and second bit costs.
 9. A computer-implemented method forvideo coding comprising: determining a pattern to provide a sequence ofneighboring blocks of a block of video data, wherein the patterncomprises a substantially spiral shape; scanning original motion vectorsassociated with the neighboring blocks according to the sequence for afirst non-zero original motion vector; when the first non-zero originalmotion vector is determined, generating a prediction motion vectorassociated with the block as the first non-zero original motion vector;or when no non-zero original is determined, generating the predictionmotion vector associated with the block as a zero motion vector; andgenerating an output bitstream based in part on the prediction motionvector for the block.
 10. The method of claim 9, wherein the patterncomprises a shape beginning at a first neighboring block of the blockand progressing through neighboring blocks clockwise around the block.11. The method of claim 9, wherein the pattern comprises a shapebeginning at a first neighboring block to the left of and aligned withthe top of the block and progressing upwards to a second block to theleft of and above the block, then to the right along the top of theblock to a third neighboring block to the right of and above the block,then downwards to a fourth neighboring block to the right of and belowthe block, then left to a fifth neighboring block to the left of andbelow the block.
 12. The method of claim 9, wherein the patterncomprises a shape beginning at a first immediately adjacent neighboringof the block, progressing clockwise through all remaining firstimmediately adjacent neighboring blocks of the block, and thenprogressing clockwise through second neighboring blocks immediatelyadjacent to first immediately adjacent neighboring blocks.
 13. Themethod of claim 9, wherein the pattern comprises a shape beginning at afirst neighboring block to the left of and aligned with the top of theblock and progressing downwards to a second neighboring block to theleft of and aligned with the bottom of the block, then skipping to athird neighboring block above and aligned with the left of the block,then progressing right to a fourth neighboring block above and alignedwith the right side of the block, then skipping to a fifth neighboringblock to the left of above the block, then skipping to a sixthneighboring block to the right of and above the block, then skipping toa seventh neighboring block to the left of below the block, thenskipping to an eighth neighboring block to the right of and below theblock, then skipping to a ninth neighboring block to the right of andaligned with the top of the block, then progressing downward to a tenthneighboring block to the right of and aligned with the bottom of theblock.
 14. The method of claim 9, further comprising: determining asecond prediction motion vector for a second block of the video datausing a median motion vector prediction technique based on second motionvectors associated with second neighboring blocks of the second block;and when the second prediction motion vector is a zero motion vector,scanning third motion vectors associated with third neighboring blocksof the second block according to the sequence to generate a thirdprediction motion vector for the second block; or when the secondprediction motion vector is a non-zero motion vector, determining athird prediction motion vector for the second block using one of aGraham motion vector prediction technique or a Graham last block motionvector prediction technique; comparing a difference between the secondprediction motion vector and the third prediction motion vector to athreshold; and when the difference is less than the threshold, selectingthe second prediction motion vector for the second block; or when thedifference is not less than the threshold, estimating a first bit costcorresponding to the second prediction motion vector and a second bitcost corresponding to the third prediction motion vector and selectingthe second or third prediction motion vector for the second block basedon the first and second bit costs.
 15. At least one non-transitorymachine readable medium comprising a plurality of instructions that, inresponse to being executed on a device, cause the device perform videocoding by: determining a pattern to provide a sequence of neighboringblocks of a block of video data, wherein the pattern comprises asubstantially spiral shape; scanning original motion vectors associatedwith the neighboring blocks according to the sequence for a firstnon-zero original motion vector; when the first non-zero original motionvector is determined, generating a prediction motion vector associatedwith the block as the first non-zero original motion vector; or when nonon-zero original is determined, generating the prediction motion vectorassociated with the block as a zero motion vector; and generating anoutput bitstream based in part on the prediction motion vector for theblock.
 16. The non-transitory machine readable medium of claim 15,wherein the pattern comprises a shape beginning at a first neighboringblock of the block and progressing through neighboring blocks clockwisearound the block.
 17. The non-transitory machine readable medium ofclaim 15, wherein the pattern comprises a shape beginning at a firstneighboring block to the left of and aligned with the top of the blockand progressing upwards to a second block to the left of and above theblock, then to the right along the top of the block to a thirdneighboring block to the right of and above the block, then downwards toa fourth neighboring block to the right of and below the block, thenleft to a fifth neighboring block to the left of and below the block.18. The non-transitory machine readable medium of claim 15, wherein thepattern comprises a shape beginning at a first immediately adjacentneighboring of the block, progressing clockwise through all remainingfirst immediately adjacent neighboring blocks of the block, and thenprogressing clockwise through second neighboring blocks immediatelyadjacent to first immediately adjacent neighboring blocks.
 19. Thenon-transitory machine readable medium of claim 15, wherein the patterncomprises a shape beginning at a first neighboring block to the left ofand aligned with the top of the block and progressing downwards to asecond neighboring block to the left of and aligned with the bottom ofthe block, then skipping to a third neighboring block above and alignedwith the left of the block, then progressing right to a fourthneighboring block above and aligned with the right side of the block,then skipping to a fifth neighboring block to the left of above theblock, then skipping to a sixth neighboring block to the right of andabove the block, then skipping to a seventh neighboring block to theleft of below the block, then skipping to an eighth neighboring block tothe right of and below the block, then skipping to a ninth neighboringblock to the right of and aligned with the top of the block, thenprogressing downward to a tenth neighboring block to the right of andaligned with the bottom of the block.
 20. The non-transitory machinereadable medium of claim 15, comprising further instructions that, inresponse to being executed on a device, cause the device perform videocoding by: determining a second prediction motion vector for a secondblock of the video data using a median motion vector predictiontechnique based on second motion vectors associated with secondneighboring blocks of the second block; and when the second predictionmotion vector is a zero motion vector, scanning third motion vectorsassociated with third neighboring blocks of the second block accordingto the sequence to generate a third prediction motion vector for thesecond block; or when the second prediction motion vector is a non-zeromotion vector, determining a third prediction motion vector for thesecond block using one of a Graham motion vector prediction technique ora Graham last block motion vector prediction technique; comparing adifference between the second prediction motion vector and the thirdprediction motion vector to a threshold; and when the difference is lessthan the threshold, selecting the second prediction motion vector forthe second block; or when the difference is not less than the threshold,estimating a first bit cost corresponding to the second predictionmotion vector and a second bit cost corresponding to the thirdprediction motion vector and selecting the second or third predictionmotion vector for the second block based on the first and second bitcosts.